[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:40 dist-test-slave-wjw3 krb5kdc[8295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:40 dist-test-slave-wjw3 krb5kdc[8295](info): set up 2 sockets
Jun 20 07:06:40 dist-test-slave-wjw3 krb5kdc[8295](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 07:06:41.299861 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:41 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:41.412762 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39611
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:41.519572 8308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:41.519843 8308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:41.519891 8308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:41.523819 8308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:41.523874 8308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:41.523892 8308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:41.523906 8308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:41.523926 8308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:41.529170 8308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:41.530189 8308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:41.531181 8308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:41.539188 8316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:41.540356 8308 server_base.cc:1034] running on GCE node
W20260620 07:06:41.540364 8314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:41.540442 8313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:41.540989 8308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:41.542059 8308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:41.543211 8308 hybrid_clock.cc:648] HybridClock initialized: now 1781939201543193 us; error 31 us; skew 500 ppm
Jun 20 07:06:41 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939201, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:41.548811 8308 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:41.549785 8308 webserver.cc:460] Webserver started at http://127.8.24.62:46453/ using document root <none> and password file <none>
I20260620 07:06:41.550321 8308 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:41.550379 8308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:41.550552 8308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:41.552661 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "c56354ed23994a22979858de708bd8af"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "39d6bb9a69e55d197f603ad801f0a893"
server_key_iv: "2cf5d48ed793a70618b8d889e4317155"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.553188 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "c56354ed23994a22979858de708bd8af"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "39d6bb9a69e55d197f603ad801f0a893"
server_key_iv: "2cf5d48ed793a70618b8d889e4317155"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.556926 8308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 07:06:41.559252 8323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:41.560587 8308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:06:41.560715 8308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "c56354ed23994a22979858de708bd8af"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "39d6bb9a69e55d197f603ad801f0a893"
server_key_iv: "2cf5d48ed793a70618b8d889e4317155"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.560817 8308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:41.592564 8308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:41.595595 8308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:41.595770 8308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:41.602402 8308 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39611
I20260620 07:06:41.602432 8375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39611 every 8 connection(s)
I20260620 07:06:41.603371 8308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:41.606081 8376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:41.608952 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8308
I20260620 07:06:41.609118 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:41.610766 8288 external_mini_cluster.cc:1428] Setting key 13fc91b043cf7733554a10f22bda82b9
I20260620 07:06:41.612753 8376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af: Bootstrap starting.
I20260620 07:06:41.615496 8376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:41.616374 8376 log.cc:824] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:41.618649 8376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af: No bootstrap required, opened a new log
I20260620 07:06:41.623261 8376 raft_consensus.cc:348] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } }
I20260620 07:06:41.623569 8376 raft_consensus.cc:374] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:41.623634 8376 raft_consensus.cc:729] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56354ed23994a22979858de708bd8af, State: Initialized, Role: FOLLOWER
I20260620 07:06:41.624076 8376 consensus_queue.cc:260] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } }
I20260620 07:06:41.624202 8376 raft_consensus.cc:388] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:41.624261 8376 raft_consensus.cc:482] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:41.624356 8376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:41.625438 8376 raft_consensus.cc:504] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } }
I20260620 07:06:41.625797 8376 leader_election.cc:302] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c56354ed23994a22979858de708bd8af; no voters:
I20260620 07:06:41.626133 8376 leader_election.cc:288] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:41.626224 8381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:41.626446 8381 raft_consensus.cc:686] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [term 1 LEADER]: Becoming Leader. State: Replica: c56354ed23994a22979858de708bd8af, State: Running, Role: LEADER
I20260620 07:06:41.626760 8381 consensus_queue.cc:237] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } }
I20260620 07:06:41.627360 8376 sys_catalog.cc:564] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:41.627978 8382 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [sys.catalog]: SysCatalogTable state changed. Reason: New leader c56354ed23994a22979858de708bd8af. Latest consensus state: current_term: 1 leader_uuid: "c56354ed23994a22979858de708bd8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } } }
I20260620 07:06:41.628095 8382 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:41.628381 8383 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c56354ed23994a22979858de708bd8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56354ed23994a22979858de708bd8af" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39611 } } }
I20260620 07:06:41.628466 8383 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:41.628737 8387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 20 07:06:41 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:41.632421 8387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:41.637852 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:41.614501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.8.24.62:39611)
0620 07:06:41.615003 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:41.615023 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:41.616061 (+ 1038us) server_negotiation.cc:407] Connection header received
0620 07:06:41.620002 (+ 3941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:41.620023 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:41.620353 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:41.620620 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:41.623559 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.624728 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.626390 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.627124 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.627305 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:41.632180 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:41.632186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:41.632201 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:41.632236 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:41.634502 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:41.635037 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:41.635043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:41.635049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:41.635158 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:41.635500 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:41.635503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:41.635505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:41.635896 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:41.636068 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:41.636641 (+ 573us) server_negotiation.cc:299] Negotiation successful
0620 07:06:41.636902 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":147,"threads_started":1}
I20260620 07:06:41.638890 8387 catalog_manager.cc:1347] Generated new cluster ID: bc61ce5cd9184bd0a9c7e498a0e0c4b3
I20260620 07:06:41.639019 8387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:41.661598 8387 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:41.662544 8387 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:41.669589 8387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c56354ed23994a22979858de708bd8af: Generated new TSK 0
I20260620 07:06:41.670295 8387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:06:41.700485 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:41.809396 8402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:41.809710 8402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:41.809756 8402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:41.813835 8402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:41.813905 8402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:41.813982 8402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:06:41.819263 8402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:41.820336 8402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:41.821331 8402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:41.830418 8408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:41.830456 8410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:41.830559 8402 server_base.cc:1034] running on GCE node
W20260620 07:06:41.830461 8407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:41.831050 8402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:41.831688 8402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:41.832854 8402 hybrid_clock.cc:648] HybridClock initialized: now 1781939201832828 us; error 35 us; skew 500 ppm
Jun 20 07:06:41 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939201, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:41.835404 8402 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:06:41.836365 8402 webserver.cc:460] Webserver started at http://127.8.24.1:40047/ using document root <none> and password file <none>
I20260620 07:06:41.836905 8402 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:41.836962 8402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:41.837138 8402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:41.839118 8402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "1f59e7f47012475ab049b372ea2628ea"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "e98a93e63c06dfc04f9ffb8c1b998cd8"
server_key_iv: "7e5deb7c2bfd6bcf843e7992c134758b"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.839663 8402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "1f59e7f47012475ab049b372ea2628ea"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "e98a93e63c06dfc04f9ffb8c1b998cd8"
server_key_iv: "7e5deb7c2bfd6bcf843e7992c134758b"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.843277 8402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:06:41.845635 8417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:41.846835 8402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:41.846930 8402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "1f59e7f47012475ab049b372ea2628ea"
format_stamp: "Formatted at 2026-06-20 07:06:41 on dist-test-slave-wjw3"
server_key: "e98a93e63c06dfc04f9ffb8c1b998cd8"
server_key_iv: "7e5deb7c2bfd6bcf843e7992c134758b"
server_key_version: "encryptionkey@0"
I20260620 07:06:41.847019 8402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:41.872172 8402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:41.875176 8402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:41.875507 8402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:41.876134 8402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:41.876972 8402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:41.877027 8402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:41.877079 8402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:41.877102 8402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:41.885752 8402 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:38769
I20260620 07:06:41.885766 8530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:38769 every 8 connection(s)
I20260620 07:06:41.886619 8402 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
Jun 20 07:06:41 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939201, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:41.896347 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8402
I20260620 07:06:41.896442 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:06:41.896713 8288 external_mini_cluster.cc:1428] Setting key c3a0b9cc162cf5ea65b5d1a631b3a6f2
I20260620 07:06:41.899777 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:41.888440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56019 (local address 127.8.24.62:39611)
0620 07:06:41.888580 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:41.888584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:41.889423 (+ 839us) server_negotiation.cc:407] Connection header received
0620 07:06:41.890753 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:41.890756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:41.890815 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:41.890887 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:41.892109 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.892638 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.893735 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.893969 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.894056 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:41.896302 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:41.896305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:41.896306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:41.896330 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:41.897732 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:41.898276 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:41.898279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:41.898280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:41.898354 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:41.898692 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:41.898694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:41.898695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:41.898869 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:41.898942 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:41.899499 (+ 557us) server_negotiation.cc:299] Negotiation successful
0620 07:06:41.899603 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:06:41.900620 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:41.888819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39611 (local address 127.8.24.1:56019)
0620 07:06:41.889273 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:41.889318 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:41.890530 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:41.891075 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:41.891082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:41.891472 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:41.891970 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:41.891983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.892749 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.892752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:41.893619 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:41.893628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:41.894115 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:41.894120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:41.894281 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:41.894781 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:41.894797 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:41.896146 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:41.897845 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:41.897850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:41.897861 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:41.898182 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:41.898435 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:41.898438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:41.898440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:41.898613 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:41.898955 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:41.898961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:41.899249 (+ 288us) client_negotiation.cc:769] Sending connection context
0620 07:06:41.899459 (+ 210us) client_negotiation.cc:240] Negotiation successful
0620 07:06:41.899823 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":113,"threads_started":1}
I20260620 07:06:41.901816 8531 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39611
I20260620 07:06:41.902032 8531 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:41.902478 8531 heartbeater.cc:511] Master 127.8.24.62:39611 requested a full tablet report, sending...
I20260620 07:06:41.903890 8340 ts_manager.cc:194] Registered new tserver with Master: 1f59e7f47012475ab049b372ea2628ea (127.8.24.1:38769)
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:06:41.904973 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:56019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:06:41.919956 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:42.027988 8536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:42.028263 8536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:42.028331 8536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:42.032398 8536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:42.032449 8536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:42.032523 8536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:06:42.037731 8536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:42.038774 8536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:42.039814 8536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:42.048803 8541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:42.048806 8542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:42.048870 8536 server_base.cc:1034] running on GCE node
W20260620 07:06:42.048803 8544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:42.049443 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:42.050045 8536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:42.051206 8536 hybrid_clock.cc:648] HybridClock initialized: now 1781939202051189 us; error 28 us; skew 500 ppm
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:42.053853 8536 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:06:42.054768 8536 webserver.cc:460] Webserver started at http://127.8.24.2:41265/ using document root <none> and password file <none>
I20260620 07:06:42.055301 8536 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:42.055361 8536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:42.055585 8536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:42.057575 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "86a2dfcb9e4e48f38bd9f79791e3c9c0"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "b65d4cb2e27d7c9f33edf3a377b1a5ba"
server_key_iv: "edc2812f8fc3a79b0f8010cd279db0be"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.058106 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "86a2dfcb9e4e48f38bd9f79791e3c9c0"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "b65d4cb2e27d7c9f33edf3a377b1a5ba"
server_key_iv: "edc2812f8fc3a79b0f8010cd279db0be"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.061715 8536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:06:42.063993 8551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.065160 8536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:42.065279 8536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "86a2dfcb9e4e48f38bd9f79791e3c9c0"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "b65d4cb2e27d7c9f33edf3a377b1a5ba"
server_key_iv: "edc2812f8fc3a79b0f8010cd279db0be"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.065372 8536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:42.083995 8536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:42.086704 8536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:42.086869 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:42.087430 8536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:42.088253 8536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:42.088297 8536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.088339 8536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:42.088367 8536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.096949 8536 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:44931
I20260620 07:06:42.096972 8664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:44931 every 8 connection(s)
I20260620 07:06:42.097795 8536 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:06:42.105438 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8536
I20260620 07:06:42.105535 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:06:42.105777 8288 external_mini_cluster.cc:1428] Setting key 9c776698c85756b519c7d9895d9b8f90
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:42.110275 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.099522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58329 (local address 127.8.24.62:39611)
0620 07:06:42.099645 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:42.099649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:42.100468 (+ 819us) server_negotiation.cc:407] Connection header received
0620 07:06:42.101689 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:42.101692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:42.101737 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:42.101808 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:42.102879 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.103389 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.104451 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.104655 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.104721 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.106999 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:42.107003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:42.107006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:42.107031 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:42.108371 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.108874 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.108877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.108879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.108972 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.109237 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.109240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.109241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.109461 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:42.109557 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:42.109992 (+ 435us) server_negotiation.cc:299] Negotiation successful
0620 07:06:42.110117 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:06:42.111035 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.099892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39611 (local address 127.8.24.2:58329)
0620 07:06:42.100311 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:42.100357 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:42.101510 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:42.101955 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:42.101961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:42.102294 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:42.102770 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:42.102782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.103511 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.103514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:42.104362 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:42.104371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.104756 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.104759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:42.104889 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.105420 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:42.105436 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:42.106844 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:42.108488 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:42.108495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:42.108511 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:42.108778 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:42.109043 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:42.109045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:42.109046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:42.109167 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:42.109545 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:42.109549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:42.109770 (+ 221us) client_negotiation.cc:769] Sending connection context
0620 07:06:42.109919 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 07:06:42.110168 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260620 07:06:42.112093 8665 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39611
I20260620 07:06:42.112308 8665 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:42.112746 8665 heartbeater.cc:511] Master 127.8.24.62:39611 requested a full tablet report, sending...
I20260620 07:06:42.113684 8340 ts_manager.cc:194] Registered new tserver with Master: 86a2dfcb9e4e48f38bd9f79791e3c9c0 (127.8.24.2:44931)
I20260620 07:06:42.114217 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:58329
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:06:42.130484 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:42.239115 8670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:42.239389 8670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:42.239473 8670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:42.243530 8670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:42.243606 8670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:42.243696 8670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:06:42.249045 8670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39611
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:42.250161 8670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:42.251199 8670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:42.260519 8675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:42.260607 8670 server_base.cc:1034] running on GCE node
W20260620 07:06:42.260486 8676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:42.260485 8678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:42.261178 8670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:42.261754 8670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:42.262902 8670 hybrid_clock.cc:648] HybridClock initialized: now 1781939202262875 us; error 41 us; skew 500 ppm
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:42.265413 8670 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:06:42.266321 8670 webserver.cc:460] Webserver started at http://127.8.24.3:41663/ using document root <none> and password file <none>
I20260620 07:06:42.266849 8670 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:42.266912 8670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:42.267094 8670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:42.269184 8670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "8263f83fb2c1479b9c09f0b9834791c3"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "cfed71f2dd86a3a22b72d559cfc5a04a"
server_key_iv: "054962cab6f6270a7f70f4971a42829a"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.269714 8670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "8263f83fb2c1479b9c09f0b9834791c3"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "cfed71f2dd86a3a22b72d559cfc5a04a"
server_key_iv: "054962cab6f6270a7f70f4971a42829a"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.273291 8670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:06:42.275576 8685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.276693 8670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:42.276809 8670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "8263f83fb2c1479b9c09f0b9834791c3"
format_stamp: "Formatted at 2026-06-20 07:06:42 on dist-test-slave-wjw3"
server_key: "cfed71f2dd86a3a22b72d559cfc5a04a"
server_key_iv: "054962cab6f6270a7f70f4971a42829a"
server_key_version: "encryptionkey@0"
I20260620 07:06:42.276895 8670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:42.292222 8670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:42.294926 8670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:42.295087 8670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:42.295722 8670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:42.296561 8670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:42.296604 8670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.296656 8670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:42.296685 8670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:42.305042 8670 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:42279
I20260620 07:06:42.305066 8798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:42279 every 8 connection(s)
I20260620 07:06:42.305918 8670 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:42.315938 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8670
I20260620 07:06:42.316049 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:06:42.316326 8288 external_mini_cluster.cc:1428] Setting key e5c75bd8f7ac89880158ff73e5ef8a60
I20260620 07:06:42.318519 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.307680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41865 (local address 127.8.24.62:39611)
0620 07:06:42.307827 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:42.307831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:42.308673 (+ 842us) server_negotiation.cc:407] Connection header received
0620 07:06:42.309895 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:42.309898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:42.309945 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:42.310019 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:42.311103 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.311642 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.312726 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.312941 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.313013 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.315168 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:42.315171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:42.315173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:42.315197 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:42.316542 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.317024 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.317027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.317028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.317111 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.317460 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.317463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.317465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.317682 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:42.317767 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:42.318288 (+ 521us) server_negotiation.cc:299] Negotiation successful
0620 07:06:42.318380 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:06:42.319407 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.308103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39611 (local address 127.8.24.3:41865)
0620 07:06:42.308532 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:42.308571 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:42.309712 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:42.310175 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:42.310181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:42.310530 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:42.310995 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:42.311008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.311728 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.311730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:42.312632 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:42.312641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.313048 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.313050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:42.313179 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.313673 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:42.313688 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:42.315021 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:42.316638 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:42.316643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:42.316654 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:42.316929 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:42.317199 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:42.317201 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:42.317203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:42.317366 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:42.317778 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:42.317785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:42.318058 (+ 273us) client_negotiation.cc:769] Sending connection context
0620 07:06:42.318248 (+ 190us) client_negotiation.cc:240] Negotiation successful
0620 07:06:42.318563 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260620 07:06:42.320370 8799 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39611
I20260620 07:06:42.320556 8799 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:42.320968 8799 heartbeater.cc:511] Master 127.8.24.62:39611 requested a full tablet report, sending...
I20260620 07:06:42.321856 8340 ts_manager.cc:194] Registered new tserver with Master: 8263f83fb2c1479b9c09f0b9834791c3 (127.8.24.3:42279)
I20260620 07:06:42.322374 8340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:41865
I20260620 07:06:42.330319 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:42.354066 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.345511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.8.24.1:38769)
0620 07:06:42.345766 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:42.345780 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:42.345806 (+ 26us) server_negotiation.cc:407] Connection header received
0620 07:06:42.345943 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:42.345948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:42.346138 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:42.346264 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:42.346898 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.347715 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.348544 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.348917 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.349008 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.351237 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:42.351243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:42.351255 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:42.351278 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:42.352613 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.352968 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.352972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.352973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.353040 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.353213 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.353215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.353216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.353420 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:42.353583 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:42.353695 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 07:06:42.353906 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
W20260620 07:06:42.354992 8445 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36300
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:42.377556 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:42.370527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.8.24.1:38769)
0620 07:06:42.370657 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:42.370661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:42.370791 (+ 130us) server_negotiation.cc:407] Connection header received
0620 07:06:42.370908 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:42.370910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:42.370950 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:42.371007 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:42.371696 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.372196 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.372919 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:42.373130 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:42.373197 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:42.375228 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:42.375231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:42.375234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:42.375255 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:42.376394 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.376715 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.376718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.376719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.376778 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:42.376947 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:42.376949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:42.376950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:42.377086 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:42.377172 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:42.377325 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 07:06:42.377411 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:06:42.378484 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8402
I20260620 07:06:42.384672 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8536
I20260620 07:06:42.390686 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8670
I20260620 07:06:42.395255 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8308
2026-06-20T07:06:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2052 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8813](info): set up 2 sockets
Jun 20 07:06:42 dist-test-slave-wjw3 krb5kdc[8813](info): commencing operation
krb5kdc: starting...
W20260620 07:06:43.354900 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:43 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:43.468398 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35627
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:43.576439 8826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:43.576731 8826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:43.576771 8826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:43.580927 8826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:43.580989 8826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:43.581009 8826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:43.581039 8826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:43.581065 8826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:43.586493 8826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:43.587610 8826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:43.588646 8826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:43.596909 8834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:43.596917 8831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:43.597012 8826 server_base.cc:1034] running on GCE node
W20260620 07:06:43.596948 8832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:43.597689 8826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:43.598721 8826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:43.599862 8826 hybrid_clock.cc:648] HybridClock initialized: now 1781939203599839 us; error 36 us; skew 500 ppm
Jun 20 07:06:43 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:43.602396 8826 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:43.603246 8826 webserver.cc:460] Webserver started at http://127.8.24.62:34635/ using document root <none> and password file <none>
I20260620 07:06:43.603817 8826 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:43.603878 8826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:43.604066 8826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:43.606053 8826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "d327b02a51f34603bc249a7730a7fd1f"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "66a4f5e61e5ec4dfa7be489730d2b7e4"
server_key_iv: "77d912270d2156f5c5c6e2c4b0009c9f"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.606585 8826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "d327b02a51f34603bc249a7730a7fd1f"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "66a4f5e61e5ec4dfa7be489730d2b7e4"
server_key_iv: "77d912270d2156f5c5c6e2c4b0009c9f"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.610380 8826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 07:06:43.612845 8841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:43.614107 8826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:43.614238 8826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "d327b02a51f34603bc249a7730a7fd1f"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "66a4f5e61e5ec4dfa7be489730d2b7e4"
server_key_iv: "77d912270d2156f5c5c6e2c4b0009c9f"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.614347 8826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:43.630883 8826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:43.634097 8826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:43.634291 8826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:43.640940 8826 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35627
I20260620 07:06:43.640972 8893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35627 every 8 connection(s)
I20260620 07:06:43.641862 8826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:43.643728 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8826
I20260620 07:06:43.643818 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:43.644048 8288 external_mini_cluster.cc:1428] Setting key 4c8edfcc3474eef58d9462bd1af89dce
I20260620 07:06:43.644646 8894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:43.651248 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f: Bootstrap starting.
Jun 20 07:06:43 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:43.653687 8894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:43.654383 8894 log.cc:824] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:43.656476 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f: No bootstrap required, opened a new log
I20260620 07:06:43.657783 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:43.645295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.8.24.62:35627)
0620 07:06:43.645701 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:43.645719 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:43.645762 (+ 43us) server_negotiation.cc:407] Connection header received
0620 07:06:43.646600 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:43.646623 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:43.647046 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:43.647421 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:43.648066 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.649498 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.650279 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.650935 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.651088 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:43.653563 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:43.653568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:43.653580 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:43.653607 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:43.655138 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:43.655600 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:43.655604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:43.655609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:43.655713 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:43.655976 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:43.655978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:43.655979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:43.656457 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:43.656635 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:43.656861 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 07:06:43.657087 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260620 07:06:43.660130 8894 raft_consensus.cc:348] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } }
I20260620 07:06:43.660367 8894 raft_consensus.cc:374] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:43.660422 8894 raft_consensus.cc:729] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d327b02a51f34603bc249a7730a7fd1f, State: Initialized, Role: FOLLOWER
I20260620 07:06:43.660805 8894 consensus_queue.cc:260] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } }
I20260620 07:06:43.660928 8894 raft_consensus.cc:388] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:43.660986 8894 raft_consensus.cc:482] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:43.661077 8894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:43.661934 8894 raft_consensus.cc:504] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } }
I20260620 07:06:43.662263 8894 leader_election.cc:302] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d327b02a51f34603bc249a7730a7fd1f; no voters:
I20260620 07:06:43.662597 8894 leader_election.cc:288] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:43.662696 8899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:43.662902 8899 raft_consensus.cc:686] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [term 1 LEADER]: Becoming Leader. State: Replica: d327b02a51f34603bc249a7730a7fd1f, State: Running, Role: LEADER
I20260620 07:06:43.663247 8899 consensus_queue.cc:237] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } }
I20260620 07:06:43.663889 8894 sys_catalog.cc:564] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:43.664415 8900 sys_catalog.cc:455] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d327b02a51f34603bc249a7730a7fd1f. Latest consensus state: current_term: 1 leader_uuid: "d327b02a51f34603bc249a7730a7fd1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } } }
I20260620 07:06:43.664609 8900 sys_catalog.cc:458] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:43.664855 8901 sys_catalog.cc:455] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d327b02a51f34603bc249a7730a7fd1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d327b02a51f34603bc249a7730a7fd1f" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35627 } } }
I20260620 07:06:43.664969 8901 sys_catalog.cc:458] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:43.665333 8907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:06:43.668517 8907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:43.673334 8907 catalog_manager.cc:1347] Generated new cluster ID: 0c726ee7459946f2b11f1d52d0f89256
I20260620 07:06:43.673400 8907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:43.691771 8907 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:43.693076 8907 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:43.712299 8907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d327b02a51f34603bc249a7730a7fd1f: Generated new TSK 0
I20260620 07:06:43.713052 8907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:06:43.743914 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:43.853555 8920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:43.853826 8920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:43.853884 8920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:43.857933 8920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:43.857997 8920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:43.858078 8920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:06:43.863253 8920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:43.864365 8920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:43.865447 8920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:43.875488 8925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:43.875519 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:43.875424 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:43.875582 8920 server_base.cc:1034] running on GCE node
I20260620 07:06:43.876070 8920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:43.876703 8920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:43.877856 8920 hybrid_clock.cc:648] HybridClock initialized: now 1781939203877833 us; error 43 us; skew 500 ppm
Jun 20 07:06:43 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:43.880407 8920 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:06:43.881353 8920 webserver.cc:460] Webserver started at http://127.8.24.1:44403/ using document root <none> and password file <none>
I20260620 07:06:43.881893 8920 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:43.881958 8920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:43.882160 8920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:43.884230 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "eea085496ff540a98c09f99988cb3626"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "6ca406ab4b0901271f24a5693c730886"
server_key_iv: "d65c707aa115e2966d91a37c7510c1ef"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.884775 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "eea085496ff540a98c09f99988cb3626"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "6ca406ab4b0901271f24a5693c730886"
server_key_iv: "d65c707aa115e2966d91a37c7510c1ef"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.888413 8920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:06:43.890728 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:43.891929 8920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:43.892032 8920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "eea085496ff540a98c09f99988cb3626"
format_stamp: "Formatted at 2026-06-20 07:06:43 on dist-test-slave-wjw3"
server_key: "6ca406ab4b0901271f24a5693c730886"
server_key_iv: "d65c707aa115e2966d91a37c7510c1ef"
server_key_version: "encryptionkey@0"
I20260620 07:06:43.892125 8920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:43.913775 8920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:43.916716 8920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:43.916903 8920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:43.917537 8920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:43.918391 8920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:43.918439 8920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:43.918493 8920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:43.918521 8920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:43.926734 8920 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:34013
I20260620 07:06:43.926755 9048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:34013 every 8 connection(s)
I20260620 07:06:43.927691 8920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:06:43.929764 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 8920
I20260620 07:06:43.929852 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:06:43.930086 8288 external_mini_cluster.cc:1428] Setting key 468e2c8161232b0d350e8f43165922ac
Jun 20 07:06:43 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:06:43.941322 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:43.929421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41189 (local address 127.8.24.62:35627)
0620 07:06:43.929564 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:43.929569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:43.930504 (+ 935us) server_negotiation.cc:407] Connection header received
0620 07:06:43.932044 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:43.932047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:43.932088 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:43.932152 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:43.933522 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.934319 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.935490 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.935845 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.935951 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:43.938172 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:43.938176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:43.938179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:43.938208 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:43.939542 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:43.939982 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:43.939985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:43.939986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:43.940064 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:43.940338 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:43.940340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:43.940341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:43.940505 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:43.940572 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:43.941046 (+ 474us) server_negotiation.cc:299] Negotiation successful
0620 07:06:43.941154 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:06:43.942281 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:43.929838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35627 (local address 127.8.24.1:41189)
0620 07:06:43.930346 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:43.930402 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:43.931818 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:43.932334 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:43.932343 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:43.932796 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:43.933388 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:43.933401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.934418 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.934420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:43.935349 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:43.935359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:43.935982 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:43.935985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:43.936122 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:43.936650 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:43.936666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:43.938007 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:43.939637 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:43.939642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:43.939654 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:43.939901 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:43.940135 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:43.940138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:43.940139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:43.940269 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:43.940573 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:43.940578 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:43.940828 (+ 250us) client_negotiation.cc:769] Sending connection context
0620 07:06:43.941007 (+ 179us) client_negotiation.cc:240] Negotiation successful
0620 07:06:43.941256 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":125,"threads_started":1}
I20260620 07:06:43.943403 9049 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35627
I20260620 07:06:43.943651 9049 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:43.944123 9049 heartbeater.cc:511] Master 127.8.24.62:35627 requested a full tablet report, sending...
I20260620 07:06:43.945637 8858 ts_manager.cc:194] Registered new tserver with Master: eea085496ff540a98c09f99988cb3626 (127.8.24.1:34013)
I20260620 07:06:43.946748 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:41189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:06:43.955343 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:44.066903 9054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:44.067201 9054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:44.067266 9054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:44.071362 9054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:44.071453 9054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:44.071542 9054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:06:44.076879 9054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:44.077942 9054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:44.078969 9054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:44.088459 9059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:44.088452 9062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:44.088452 9060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:44.088529 9054 server_base.cc:1034] running on GCE node
I20260620 07:06:44.089006 9054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:44.089617 9054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:44.090763 9054 hybrid_clock.cc:648] HybridClock initialized: now 1781939204090748 us; error 35 us; skew 500 ppm
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:44.093411 9054 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:06:44.094326 9054 webserver.cc:460] Webserver started at http://127.8.24.2:44053/ using document root <none> and password file <none>
I20260620 07:06:44.094909 9054 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:44.094977 9054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:44.095176 9054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:44.097302 9054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "9829db0ae3714a6aa4ff12ff85790608"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "7b81b2009dee6fdf7ddeb65b23977953"
server_key_iv: "ebc708375024dada0df7baf9733d644c"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.097857 9054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "9829db0ae3714a6aa4ff12ff85790608"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "7b81b2009dee6fdf7ddeb65b23977953"
server_key_iv: "ebc708375024dada0df7baf9733d644c"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.101588 9054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 07:06:44.103960 9069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.105135 9054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:44.105242 9054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "9829db0ae3714a6aa4ff12ff85790608"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "7b81b2009dee6fdf7ddeb65b23977953"
server_key_iv: "ebc708375024dada0df7baf9733d644c"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.105336 9054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:44.123754 9054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:44.126745 9054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:44.126928 9054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:44.127562 9054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:44.128394 9054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:44.128441 9054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.128494 9054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:44.128520 9054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.136852 9054 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:46803
I20260620 07:06:44.136885 9182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:46803 every 8 connection(s)
I20260620 07:06:44.137758 9054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:06:44.140831 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9054
I20260620 07:06:44.140935 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:06:44.141167 8288 external_mini_cluster.cc:1428] Setting key 51ab982ab7c445f557f49c7109bd5379
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:06:44.151386 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.139550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:36643 (local address 127.8.24.62:35627)
0620 07:06:44.139682 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.139686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.140603 (+ 917us) server_negotiation.cc:407] Connection header received
0620 07:06:44.142057 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.142061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.142117 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.142190 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.143479 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.143996 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.145127 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.145347 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.145432 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.147895 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.147899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.147901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.147927 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.149347 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.149830 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.149834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.149835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.149913 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.150262 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.150265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.150267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.150462 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.150547 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.151087 (+ 540us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.151222 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:06:44.152493 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.139952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35627 (local address 127.8.24.2:36643)
0620 07:06:44.140438 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.140486 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.141843 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.142409 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.142420 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.142807 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:44.143299 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.143322 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.144112 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.144115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.145005 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.145014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.145485 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.145489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.145653 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.146229 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:44.146245 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:44.147719 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:44.149451 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:44.149457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:44.149476 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:44.149729 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:44.149991 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:44.149994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:44.149996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:44.150175 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:44.150558 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:44.150566 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:44.150829 (+ 263us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.151040 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.151455 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260620 07:06:44.153607 9183 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35627
I20260620 07:06:44.153883 9183 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:44.154470 9183 heartbeater.cc:511] Master 127.8.24.62:35627 requested a full tablet report, sending...
I20260620 07:06:44.155601 8858 ts_manager.cc:194] Registered new tserver with Master: 9829db0ae3714a6aa4ff12ff85790608 (127.8.24.2:46803)
I20260620 07:06:44.156373 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:36643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:06:44.166699 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:44.276759 9188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:44.277042 9188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:44.277086 9188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:44.281055 9188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:44.281116 9188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:44.281176 9188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:06:44.286949 9188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35627
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:44.288053 9188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:44.289059 9188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:44.298693 9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:44.298707 9196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:44.298727 9193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:44.299065 9188 server_base.cc:1034] running on GCE node
I20260620 07:06:44.299512 9188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:44.300133 9188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:44.301280 9188 hybrid_clock.cc:648] HybridClock initialized: now 1781939204301270 us; error 42 us; skew 500 ppm
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:44.303841 9188 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:06:44.304682 9188 webserver.cc:460] Webserver started at http://127.8.24.3:40261/ using document root <none> and password file <none>
I20260620 07:06:44.305225 9188 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:44.305289 9188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:44.305490 9188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:44.307494 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "77cdfcb617374e1b9380ea578904e64a"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "b4ea86cd5167adfa71850469d7598976"
server_key_iv: "df47b6940374498595b7921b2933bdcc"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.308053 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "77cdfcb617374e1b9380ea578904e64a"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "b4ea86cd5167adfa71850469d7598976"
server_key_iv: "df47b6940374498595b7921b2933bdcc"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.311642 9188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260620 07:06:44.313920 9203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.315119 9188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:44.315220 9188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "77cdfcb617374e1b9380ea578904e64a"
format_stamp: "Formatted at 2026-06-20 07:06:44 on dist-test-slave-wjw3"
server_key: "b4ea86cd5167adfa71850469d7598976"
server_key_iv: "df47b6940374498595b7921b2933bdcc"
server_key_version: "encryptionkey@0"
I20260620 07:06:44.315308 9188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:44.338680 9188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:44.341743 9188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:44.341928 9188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:44.342545 9188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:44.343379 9188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:44.343426 9188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.343530 9188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:44.343561 9188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:44.351753 9188 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36929
I20260620 07:06:44.351779 9316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36929 every 8 connection(s)
I20260620 07:06:44.352618 9188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:44.362566 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9188
I20260620 07:06:44.362672 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:06:44.362963 8288 external_mini_cluster.cc:1428] Setting key 9ec0ace77b4d87d05baf2e43fd73a35c
I20260620 07:06:44.365801 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.354390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53937 (local address 127.8.24.62:35627)
0620 07:06:44.354512 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.354515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.355340 (+ 825us) server_negotiation.cc:407] Connection header received
0620 07:06:44.356676 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.356679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.356739 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.356809 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.358041 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.358560 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.359713 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.359937 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.360024 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.362278 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.362281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.362284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.362331 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.363803 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.364258 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.364262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.364264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.364342 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.364691 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.364694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.364696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.364941 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.365052 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.365527 (+ 475us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.365646 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:06:44.366668 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.354747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35627 (local address 127.8.24.3:53937)
0620 07:06:44.355190 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.355233 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.356456 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.356995 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.357003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.357367 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:44.357902 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.357916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.358676 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.358678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.359601 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.359610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.360072 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.360076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.360242 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.360787 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:44.360803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:44.362121 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:44.363901 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:44.363907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:44.363924 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:44.364168 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:44.364418 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:44.364421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:44.364423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:44.364601 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:44.365052 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:44.365058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:44.365302 (+ 244us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.365473 (+ 171us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.365793 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
I20260620 07:06:44.367699 9317 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35627
I20260620 07:06:44.367905 9317 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:44.368372 9317 heartbeater.cc:511] Master 127.8.24.62:35627 requested a full tablet report, sending...
I20260620 07:06:44.369330 8858 ts_manager.cc:194] Registered new tserver with Master: 77cdfcb617374e1b9380ea578904e64a (127.8.24.3:36929)
I20260620 07:06:44.369876 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:53937
I20260620 07:06:44.376677 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:06:44.386505 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.380033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.8.24.62:35627)
0620 07:06:44.380156 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.380161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.380311 (+ 150us) server_negotiation.cc:407] Connection header received
0620 07:06:44.380453 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.380455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.380494 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.380554 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.381205 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.381712 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.382492 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.382701 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.382783 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.384001 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.384004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.384006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.384028 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.385042 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.385391 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.385395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.385396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.385469 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.385695 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.385698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.385700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.385920 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.386035 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.386249 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.386355 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:06:44.391060 8858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:06:44.394073 8858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:44.408215 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.402280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34013 (local address 127.0.0.1:49754)
0620 07:06:44.402658 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.402708 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.402827 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.403610 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.403615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.403635 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:44.403766 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.403772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405398 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.405401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.406603 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.406612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407702 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.407813 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.407870 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.407952 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.408060 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":87,"thread_start_us":84,"threads_started":1}
I20260620 07:06:44.408406 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.403067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36929 (local address 127.0.0.1:43492)
0620 07:06:44.403336 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.403349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.403452 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.404573 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.404576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.404591 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:44.404671 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.404676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405673 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.405676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.406902 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.406909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407877 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.408008 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.408028 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.408098 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.408184 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":119,"thread_start_us":95,"threads_started":1}
I20260620 07:06:44.408582 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.402366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:46803 (local address 127.0.0.1:43612)
0620 07:06:44.402731 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.402745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.402837 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.404025 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.404028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.404043 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:44.404128 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.404133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405455 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.405458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.406874 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.406885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407853 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.407978 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.408026 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.408098 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.408185 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":149,"thread_start_us":96,"threads_started":1}
I20260620 07:06:44.408674 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.403194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.8.24.3:36929)
0620 07:06:44.404237 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.404249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.404270 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:06:44.404337 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.404340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.404460 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.404575 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:44.404769 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405549 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407011 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407596 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407684 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.407819 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.408431 (+ 612us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.408530 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
I20260620 07:06:44.408844 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.403069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43612 (local address 127.8.24.2:46803)
0620 07:06:44.403613 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.403624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.403645 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:06:44.403718 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.403723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.403892 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.404038 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:44.404248 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405324 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.406982 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407744 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.407837 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.407969 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.408137 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.408227 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":97,"threads_started":1}
I20260620 07:06:44.409078 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.402395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.8.24.1:34013)
0620 07:06:44.403088 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.403102 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.403124 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:06:44.403199 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.403204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.403418 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.403908 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:44.404042 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.405271 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.406729 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.407568 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.408639 (+ 1071us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.408784 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.408847 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.408942 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":78,"threads_started":1}
I20260620 07:06:44.411350 9251 tablet_service.cc:1467] Processing CreateTablet for tablet 4717b579a4eb4c9c9663e136567d507a (DEFAULT_TABLE table=test-table [id=2eef72a1c1ff4df4886f846c99a293ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:44.411355 9117 tablet_service.cc:1467] Processing CreateTablet for tablet 4717b579a4eb4c9c9663e136567d507a (DEFAULT_TABLE table=test-table [id=2eef72a1c1ff4df4886f846c99a293ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:44.412272 9117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4717b579a4eb4c9c9663e136567d507a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:44.412272 9251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4717b579a4eb4c9c9663e136567d507a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:44.417049 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 4717b579a4eb4c9c9663e136567d507a (DEFAULT_TABLE table=test-table [id=2eef72a1c1ff4df4886f846c99a293ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:44.418095 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4717b579a4eb4c9c9663e136567d507a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:44.418862 9334 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Bootstrap starting.
I20260620 07:06:44.420675 9335 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Bootstrap starting.
I20260620 07:06:44.421511 9334 tablet_bootstrap.cc:654] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:44.422390 9334 log.cc:824] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:44.422935 9335 tablet_bootstrap.cc:654] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:44.424187 9335 log.cc:824] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:44.424641 9334 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: No bootstrap required, opened a new log
I20260620 07:06:44.424644 9337 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Bootstrap starting.
I20260620 07:06:44.424903 9334 ts_tablet_manager.cc:1397] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260620 07:06:44.426563 9337 tablet_bootstrap.cc:654] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:44.426986 9335 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: No bootstrap required, opened a new log
I20260620 07:06:44.427162 9335 ts_tablet_manager.cc:1397] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260620 07:06:44.427619 9337 log.cc:824] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:44.428656 9334 raft_consensus.cc:348] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.428944 9334 raft_consensus.cc:374] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:44.429028 9334 raft_consensus.cc:729] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9829db0ae3714a6aa4ff12ff85790608, State: Initialized, Role: FOLLOWER
I20260620 07:06:44.429152 9337 tablet_bootstrap.cc:492] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: No bootstrap required, opened a new log
I20260620 07:06:44.429320 9337 ts_tablet_manager.cc:1397] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:06:44.429526 9334 consensus_queue.cc:260] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.430192 9183 heartbeater.cc:503] Master 127.8.24.62:35627 was elected leader, sending a full tablet report...
I20260620 07:06:44.430563 9334 ts_tablet_manager.cc:1428] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:06:44.430809 9335 raft_consensus.cc:348] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.431006 9335 raft_consensus.cc:374] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:44.431036 9335 raft_consensus.cc:729] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77cdfcb617374e1b9380ea578904e64a, State: Initialized, Role: FOLLOWER
I20260620 07:06:44.431648 9335 consensus_queue.cc:260] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.432453 9335 ts_tablet_manager.cc:1428] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:06:44.432755 9317 heartbeater.cc:503] Master 127.8.24.62:35627 was elected leader, sending a full tablet report...
I20260620 07:06:44.432701 9337 raft_consensus.cc:348] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.432942 9337 raft_consensus.cc:374] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:44.432986 9337 raft_consensus.cc:729] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea085496ff540a98c09f99988cb3626, State: Initialized, Role: FOLLOWER
I20260620 07:06:44.433341 9337 consensus_queue.cc:260] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.434183 9337 ts_tablet_manager.cc:1428] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:06:44.434474 9049 heartbeater.cc:503] Master 127.8.24.62:35627 was elected leader, sending a full tablet report...
W20260620 07:06:44.603627 9318 tablet.cc:2369] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:06:44.638808 9184 tablet.cc:2369] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:06:44.678995 9050 tablet.cc:2369] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:44.721381 9341 raft_consensus.cc:482] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:44.721580 9341 raft_consensus.cc:504] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.722710 9341 leader_election.cc:288] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eea085496ff540a98c09f99988cb3626 (127.8.24.1:34013), 9829db0ae3714a6aa4ff12ff85790608 (127.8.24.2:46803)
I20260620 07:06:44.726287 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.722992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:46803 (local address 127.8.24.3:38851)
0620 07:06:44.723187 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.723201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.723307 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.723569 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.723572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.723588 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:44.723689 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.723694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.724367 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.724371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.725237 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.725246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.725863 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.725866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.725969 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.725991 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.726060 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.726134 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":35}
I20260620 07:06:44.726372 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.723057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38851 (local address 127.8.24.2:46803)
0620 07:06:44.723196 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.723200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.723270 (+ 70us) server_negotiation.cc:407] Connection header received
0620 07:06:44.723423 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.723426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.723488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.723552 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:44.723783 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.724269 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.725362 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.725743 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.725838 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.725901 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.726165 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.726249 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 07:06:44.726451 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.723145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:48393 (local address 127.8.24.1:34013)
0620 07:06:44.723263 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.723267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.723505 (+ 238us) server_negotiation.cc:407] Connection header received
0620 07:06:44.723636 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.723639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.723683 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.723742 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:44.723983 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.724475 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.725516 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.725942 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.726036 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.726097 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.726262 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.726334 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 07:06:44.726761 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.722992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34013 (local address 127.8.24.3:48393)
0620 07:06:44.723389 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:44.723400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:44.723534 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:44.723762 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:44.723764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:44.723778 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:44.723884 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.723889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.724575 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.724578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.725402 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:44.725410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.726038 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.726042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:44.726137 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.726159 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:06:44.726560 (+ 401us) client_negotiation.cc:240] Negotiation successful
0620 07:06:44.726632 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":52,"thread_start_us":110,"threads_started":1}
I20260620 07:06:44.726987 9137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4717b579a4eb4c9c9663e136567d507a" candidate_uuid: "77cdfcb617374e1b9380ea578904e64a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9829db0ae3714a6aa4ff12ff85790608" is_pre_election: true
I20260620 07:06:44.727329 9137 raft_consensus.cc:2454] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77cdfcb617374e1b9380ea578904e64a in term 0.
I20260620 07:06:44.727370 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4717b579a4eb4c9c9663e136567d507a" candidate_uuid: "77cdfcb617374e1b9380ea578904e64a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea085496ff540a98c09f99988cb3626" is_pre_election: true
I20260620 07:06:44.727674 9003 raft_consensus.cc:2454] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77cdfcb617374e1b9380ea578904e64a in term 0.
I20260620 07:06:44.727869 9204 leader_election.cc:302] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77cdfcb617374e1b9380ea578904e64a, 9829db0ae3714a6aa4ff12ff85790608; no voters:
I20260620 07:06:44.728147 9341 raft_consensus.cc:2781] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:44.728221 9341 raft_consensus.cc:482] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:44.728251 9341 raft_consensus.cc:3037] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:44.729218 9341 raft_consensus.cc:504] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.729557 9341 leader_election.cc:288] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [CANDIDATE]: Term 1 election: Requested vote from peers eea085496ff540a98c09f99988cb3626 (127.8.24.1:34013), 9829db0ae3714a6aa4ff12ff85790608 (127.8.24.2:46803)
I20260620 07:06:44.729873 9137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4717b579a4eb4c9c9663e136567d507a" candidate_uuid: "77cdfcb617374e1b9380ea578904e64a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9829db0ae3714a6aa4ff12ff85790608"
I20260620 07:06:44.729892 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4717b579a4eb4c9c9663e136567d507a" candidate_uuid: "77cdfcb617374e1b9380ea578904e64a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eea085496ff540a98c09f99988cb3626"
I20260620 07:06:44.730000 9137 raft_consensus.cc:3037] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:44.729998 9003 raft_consensus.cc:3037] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:44.731222 9137 raft_consensus.cc:2454] T 4717b579a4eb4c9c9663e136567d507a P 9829db0ae3714a6aa4ff12ff85790608 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77cdfcb617374e1b9380ea578904e64a in term 1.
I20260620 07:06:44.731222 9003 raft_consensus.cc:2454] T 4717b579a4eb4c9c9663e136567d507a P eea085496ff540a98c09f99988cb3626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77cdfcb617374e1b9380ea578904e64a in term 1.
I20260620 07:06:44.731552 9206 leader_election.cc:302] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77cdfcb617374e1b9380ea578904e64a, eea085496ff540a98c09f99988cb3626; no voters:
I20260620 07:06:44.731696 9341 raft_consensus.cc:2781] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:44.731915 9341 raft_consensus.cc:686] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [term 1 LEADER]: Becoming Leader. State: Replica: 77cdfcb617374e1b9380ea578904e64a, State: Running, Role: LEADER
I20260620 07:06:44.732254 9341 consensus_queue.cc:237] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } }
I20260620 07:06:44.735236 8857 catalog_manager.cc:5495] T 4717b579a4eb4c9c9663e136567d507a P 77cdfcb617374e1b9380ea578904e64a reported cstate change: term changed from 0 to 1, leader changed from <none> to 77cdfcb617374e1b9380ea578904e64a (127.8.24.3). New cstate: current_term: 1 leader_uuid: "77cdfcb617374e1b9380ea578904e64a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eea085496ff540a98c09f99988cb3626" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9829db0ae3714a6aa4ff12ff85790608" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 46803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77cdfcb617374e1b9380ea578904e64a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36929 } health_report { overall_health: HEALTHY } } }
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:44.771792 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.763279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.8.24.1:34013)
0620 07:06:44.763471 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.763475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.763604 (+ 129us) server_negotiation.cc:407] Connection header received
0620 07:06:44.763748 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.763751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.763800 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.763861 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.764560 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.765080 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.765933 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.766157 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.766251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.768615 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.768629 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.768644 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.768682 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.770105 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.770522 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.770527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.770528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.770616 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.770862 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.770865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.770867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.771112 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.771324 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.771523 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.771636 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:44.796725 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.789144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.8.24.1:34013)
0620 07:06:44.789300 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.789304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.789393 (+ 89us) server_negotiation.cc:407] Connection header received
0620 07:06:44.789521 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.789524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.789570 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.789628 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.790253 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.790796 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.791629 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.791850 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.791925 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.794081 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.794084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.794087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.794111 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.795325 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.795724 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.795726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.795728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.795798 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.796006 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.796008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.796009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.796148 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.796237 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.796481 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.796574 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:44.820376 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:44.812564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.8.24.1:34013)
0620 07:06:44.812714 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:44.812718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:44.812832 (+ 114us) server_negotiation.cc:407] Connection header received
0620 07:06:44.812992 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:44.812995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:44.813036 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:44.813102 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:44.813773 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.814282 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.815092 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:44.815301 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:44.815394 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:44.817616 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:44.817618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:44.817620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:44.817646 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:44.818929 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.819290 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.819292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.819294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.819371 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:44.819619 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:44.819621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:44.819622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:44.819780 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:44.819877 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:44.820102 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 07:06:44.820197 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260620 07:06:44.821666 8963 tablet_service.cc:2908] Rejecting scan request for tablet 4717b579a4eb4c9c9663e136567d507a: Uninitialized: clean time has not yet been initialized
I20260620 07:06:44.823483 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8920
I20260620 07:06:44.828715 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9054
I20260620 07:06:44.833799 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9188
I20260620 07:06:44.838408 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 8826
2026-06-20T07:06:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2443 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[9358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[9358](info): set up 2 sockets
Jun 20 07:06:44 dist-test-slave-wjw3 krb5kdc[9358](info): commencing operation
krb5kdc: starting...
W20260620 07:06:45.779021 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:45 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:45.902220 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39607
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:46.009087 9371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:46.009375 9371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:46.009428 9371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:46.013479 9371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:46.013550 9371 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 07:06:46.013573 9371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:46.013595 9371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:46.013616 9371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:46.013641 9371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:46.019083 9371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39607
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:46.020198 9371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:46.021212 9371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:46.029421 9377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.029451 9371 server_base.cc:1034] running on GCE node
W20260620 07:06:46.029500 9376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.029421 9379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.030162 9371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:46.031225 9371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:46.032371 9371 hybrid_clock.cc:648] HybridClock initialized: now 1781939206032351 us; error 39 us; skew 500 ppm
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:46.034946 9371 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:46.035955 9371 webserver.cc:460] Webserver started at http://127.8.24.62:35149/ using document root <none> and password file <none>
I20260620 07:06:46.036494 9371 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:46.036556 9371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:46.036759 9371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:46.038851 9371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "89a01c7a1ae747fd9209e23531029763"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4d56e25c4c95fbf0eb74115d7223a174"
server_key_iv: "6394f2fefe43775ae4ae6c50bf1469c7"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.039471 9371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "89a01c7a1ae747fd9209e23531029763"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4d56e25c4c95fbf0eb74115d7223a174"
server_key_iv: "6394f2fefe43775ae4ae6c50bf1469c7"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.043325 9371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 07:06:46.045847 9386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.047137 9371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:46.047251 9371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "89a01c7a1ae747fd9209e23531029763"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4d56e25c4c95fbf0eb74115d7223a174"
server_key_iv: "6394f2fefe43775ae4ae6c50bf1469c7"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.047363 9371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:46.108088 9371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:46.110909 9371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:46.111086 9371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:46.118613 9371 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39607
I20260620 07:06:46.118624 9448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39607 every 8 connection(s)
I20260620 07:06:46.119611 9371 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:46.122371 9449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.127831 9449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Bootstrap starting.
I20260620 07:06:46.128139 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9371
I20260620 07:06:46.128248 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:46.128505 8288 external_mini_cluster.cc:1428] Setting key 677cc87666bfd1dac15e3b7758098b5e
I20260620 07:06:46.130457 9449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.131153 9449 log.cc:824] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:46.132932 9449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: No bootstrap required, opened a new log
I20260620 07:06:46.136502 9449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } }
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:46.136739 9449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.136785 9449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a01c7a1ae747fd9209e23531029763, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.137230 9449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } }
I20260620 07:06:46.137348 9449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:46.137415 9449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:46.137506 9449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:46.138543 9449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } }
I20260620 07:06:46.138882 9449 leader_election.cc:302] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89a01c7a1ae747fd9209e23531029763; no voters:
I20260620 07:06:46.139173 9449 leader_election.cc:288] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:46.139256 9454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:46.139429 9454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [term 1 LEADER]: Becoming Leader. State: Replica: 89a01c7a1ae747fd9209e23531029763, State: Running, Role: LEADER
I20260620 07:06:46.139771 9454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } }
I20260620 07:06:46.140532 9449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:46.142201 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.129851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.8.24.62:39607)
0620 07:06:46.130258 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.130274 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.130310 (+ 36us) server_negotiation.cc:407] Connection header received
0620 07:06:46.130982 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.131001 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.131361 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.131698 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.132447 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.133459 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.134297 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.134767 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.134932 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.137324 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.137331 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.137351 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.137390 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.139106 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.139632 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.139638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.139645 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.139788 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.140068 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.140072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.140074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.140487 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.140663 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.140978 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.141270 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":116,"threads_started":1}
W20260620 07:06:46.145183 9468 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:06:46.145260 9468 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:06:46.145268 9455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89a01c7a1ae747fd9209e23531029763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } } }
I20260620 07:06:46.145409 9455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:46.145673 9456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89a01c7a1ae747fd9209e23531029763. Latest consensus state: current_term: 1 leader_uuid: "89a01c7a1ae747fd9209e23531029763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a01c7a1ae747fd9209e23531029763" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39607 } } }
I20260620 07:06:46.145766 9471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:06:46.145780 9456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:46.146529 9471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:46.152403 9471 catalog_manager.cc:1347] Generated new cluster ID: 2dd64f685ba94dc18e7318d6bfde7706
I20260620 07:06:46.152472 9471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:46.161711 9471 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:46.162662 9471 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:46.171320 9471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Generated new TSK 0
I20260620 07:06:46.172209 9471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:06:46.204080 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39607
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:46.313534 9475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:46.313803 9475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:46.313874 9475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:46.317907 9475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:46.317975 9475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:46.318030 9475 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:46.318085 9475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:06:46.323422 9475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:46.324558 9475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:46.325587 9475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:46.334856 9483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.334856 9480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.334926 9481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.335053 9475 server_base.cc:1034] running on GCE node
I20260620 07:06:46.335587 9475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:46.336215 9475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:46.337364 9475 hybrid_clock.cc:648] HybridClock initialized: now 1781939206337346 us; error 36 us; skew 500 ppm
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:46.339993 9475 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:06:46.340929 9475 webserver.cc:460] Webserver started at http://127.8.24.1:45819/ using document root <none> and password file <none>
I20260620 07:06:46.341497 9475 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:46.341562 9475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:46.341758 9475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:46.343858 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "a13f7215bfc74c96be9883b0c1b48ce1"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "8b23a8191322e975665545cd65530b9b"
server_key_iv: "07e6c4445bf66d97351b2630e4c4bb2c"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.344456 9475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "a13f7215bfc74c96be9883b0c1b48ce1"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "8b23a8191322e975665545cd65530b9b"
server_key_iv: "07e6c4445bf66d97351b2630e4c4bb2c"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.348201 9475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 07:06:46.350636 9490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.351945 9475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:46.352051 9475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "a13f7215bfc74c96be9883b0c1b48ce1"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "8b23a8191322e975665545cd65530b9b"
server_key_iv: "07e6c4445bf66d97351b2630e4c4bb2c"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.352147 9475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:46.385151 9475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:46.387974 9475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:46.388139 9475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:46.389580 9475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:46.389635 9475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.389689 9475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:46.389714 9475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:46.398993 9475 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:46823
I20260620 07:06:46.399065 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:46823 every 8 connection(s)
I20260620 07:06:46.400087 9475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:06:46.400629 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9475
I20260620 07:06:46.400738 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:06:46.401003 8288 external_mini_cluster.cc:1428] Setting key a10982333908c35f4c7f6fe74f7921b1
I20260620 07:06:46.402696 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.390344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41069 (local address 127.8.24.62:39607)
0620 07:06:46.390530 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.390535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.391385 (+ 850us) server_negotiation.cc:407] Connection header received
0620 07:06:46.392510 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.392514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.392574 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.392634 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.394341 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.395165 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.396370 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.396605 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.396694 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.399116 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.399120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.399123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.399150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.400628 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.401205 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.401209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.401211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.401297 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.401670 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.401673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.401675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.401853 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.401936 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.402418 (+ 482us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.402540 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 07:06:46.403512 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.390764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.1:41069)
0620 07:06:46.391227 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.391278 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.392244 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.393070 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.393078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.393499 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.394191 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.394212 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.395279 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.395283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.396248 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.396257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.396746 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.396750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.396928 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.397489 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.397510 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.398952 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.400776 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.400782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.400798 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.401108 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.401397 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.401401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.401403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.401571 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.401953 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.401959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.402195 (+ 236us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.402358 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.402674 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
I20260620 07:06:46.404765 9606 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39607
I20260620 07:06:46.404902 9606 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:46.405366 9606 heartbeater.cc:511] Master 127.8.24.62:39607 requested a full tablet report, sending...
I20260620 07:06:46.406759 9403 ts_manager.cc:194] Registered new tserver with Master: a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823)
I20260620 07:06:46.408331 9403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:41069
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:06:46.413822 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.406814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.1:51515)
0620 07:06:46.407087 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.407102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.407184 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.407415 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.407418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.407647 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.407752 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.407758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.408566 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.408568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.409169 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.409179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.409743 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.409747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.409860 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.410459 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.410473 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.410937 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.412512 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.412515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.412516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.412696 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.412890 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.412892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.412893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.412964 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.413306 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.413309 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.413392 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.413476 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.413615 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":29,"thread_start_us":97,"threads_started":1}
I20260620 07:06:46.413841 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.407068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:51515 (local address 127.8.24.62:39607)
0620 07:06:46.407169 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.407172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.407185 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:06:46.407270 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.407273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.407312 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.407386 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.407860 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.408421 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.409301 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.409614 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.409730 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.411074 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.411077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.411079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.411101 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.412421 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.412765 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.412767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.412769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.412832 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.413042 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.413046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.413048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.413219 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.413305 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.413541 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.413639 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:06:46.428013 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39607
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:46.534209 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:46.534477 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:46.534528 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:46.538534 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:46.538601 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:46.538647 9615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:46.538697 9615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:06:46.544095 9615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:46.545117 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:46.546113 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:46.555289 9621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.555301 9620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.555301 9623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.555631 9615 server_base.cc:1034] running on GCE node
I20260620 07:06:46.556033 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:46.556710 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:46.557855 9615 hybrid_clock.cc:648] HybridClock initialized: now 1781939206557825 us; error 43 us; skew 500 ppm
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:46.562983 9615 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:06:46.563974 9615 webserver.cc:460] Webserver started at http://127.8.24.2:46197/ using document root <none> and password file <none>
I20260620 07:06:46.564514 9615 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:46.564579 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:46.564774 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:46.566777 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "fd377f5e6119239427f06e8e85998c16"
server_key_iv: "0e4aed1560a4699260f6d52593863360"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.567304 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "fd377f5e6119239427f06e8e85998c16"
server_key_iv: "0e4aed1560a4699260f6d52593863360"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.570868 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:06:46.573221 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.574507 9615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 07:06:46.574615 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "fd377f5e6119239427f06e8e85998c16"
server_key_iv: "0e4aed1560a4699260f6d52593863360"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.574720 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:46.592156 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:46.595237 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:46.595407 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:46.596824 9615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:46.596871 9615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.596923 9615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:46.596952 9615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:46.606634 9615 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45825
I20260620 07:06:46.606699 9745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45825 every 8 connection(s)
I20260620 07:06:46.607661 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:06:46.609758 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.597676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51955 (local address 127.8.24.62:39607)
0620 07:06:46.597784 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.597787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.598831 (+ 1044us) server_negotiation.cc:407] Connection header received
0620 07:06:46.599949 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.599954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.600013 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.600069 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.601429 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.602173 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.603747 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.603976 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.604064 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.606526 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.606530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.606533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.606564 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.607925 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.608468 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.608471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.608472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.608554 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.608868 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.608870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.608871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.609051 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.609121 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.609544 (+ 423us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.609640 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:06:46.610512 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.598230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.2:51955)
0620 07:06:46.598692 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.598732 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.599688 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.600422 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.600432 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.600765 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.601284 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.601298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.602284 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.602287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.603637 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.603645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.604112 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.604116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.604291 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.604875 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.604922 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.606361 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.608022 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.608028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.608042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.608380 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.608628 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.608631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.608633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.608795 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.609122 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.609126 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.609344 (+ 218us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.609505 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.609742 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":128,"threads_started":1}
I20260620 07:06:46.611673 9746 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39607
I20260620 07:06:46.611799 9746 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:46.612260 9746 heartbeater.cc:511] Master 127.8.24.62:39607 requested a full tablet report, sending...
I20260620 07:06:46.613291 9402 ts_manager.cc:194] Registered new tserver with Master: 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825)
I20260620 07:06:46.613489 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9615
I20260620 07:06:46.613595 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:06:46.613890 8288 external_mini_cluster.cc:1428] Setting key d71d55744b3309be0dda44a4afb3a63c
I20260620 07:06:46.613970 9402 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:51955
I20260620 07:06:46.620326 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.613523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.2:41233)
0620 07:06:46.613796 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.613810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.614014 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.614373 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.614375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.614556 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.614673 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.614681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.615553 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.615556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.616106 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.616113 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.616516 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.616519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.616604 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.617058 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.617072 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.617384 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.618913 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.618915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.618917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.619132 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.619402 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.619404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.619406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.619517 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.619942 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.619944 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.620012 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.620082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.620181 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260620 07:06:46.620426 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.613837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:41233 (local address 127.8.24.62:39607)
0620 07:06:46.613942 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.613946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.613959 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:06:46.614118 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.614122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.614168 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.614255 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.614793 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.615418 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.616208 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.616425 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.616505 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.617504 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.617508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.617510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.617535 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.618819 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.619225 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.619229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.619231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.619317 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.619592 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.619595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.619596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.619860 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.619962 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.620156 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.620264 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:06:46.638115 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39607
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:46.746191 9755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:46.746464 9755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:46.746522 9755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:46.750563 9755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:46.750644 9755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:46.750696 9755 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:46.750762 9755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:06:46.756243 9755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:46.757337 9755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:46.758338 9755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:46.767518 9763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.767692 9755 server_base.cc:1034] running on GCE node
W20260620 07:06:46.768746 9761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:46.769371 9760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:46.769791 9755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:46.770452 9755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:46.771610 9755 hybrid_clock.cc:648] HybridClock initialized: now 1781939206771587 us; error 33 us; skew 500 ppm
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:46.774209 9755 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:06:46.775110 9755 webserver.cc:460] Webserver started at http://127.8.24.3:37809/ using document root <none> and password file <none>
I20260620 07:06:46.775688 9755 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:46.775750 9755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:46.775924 9755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:46.777865 9755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "dea59aeb8011487995e80c00507eb92a"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4cab2932424f0f84a314f8de058dc402"
server_key_iv: "d6c9d121c175087c7af6d3dac14976fc"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.778373 9755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "dea59aeb8011487995e80c00507eb92a"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4cab2932424f0f84a314f8de058dc402"
server_key_iv: "d6c9d121c175087c7af6d3dac14976fc"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.781877 9755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:06:46.784106 9770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.785252 9755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:46.785346 9755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "dea59aeb8011487995e80c00507eb92a"
format_stamp: "Formatted at 2026-06-20 07:06:46 on dist-test-slave-wjw3"
server_key: "4cab2932424f0f84a314f8de058dc402"
server_key_iv: "d6c9d121c175087c7af6d3dac14976fc"
server_key_version: "encryptionkey@0"
I20260620 07:06:46.785441 9755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:46.796636 9755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:46.799229 9755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:46.799391 9755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:46.800798 9755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:46.800844 9755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:46.800891 9755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:46.800921 9755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:46.810637 9755 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:38755
I20260620 07:06:46.810953 9885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:38755 every 8 connection(s)
I20260620 07:06:46.811578 9755 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:06:46.813477 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.801482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35339 (local address 127.8.24.62:39607)
0620 07:06:46.801624 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.801629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.802872 (+ 1243us) server_negotiation.cc:407] Connection header received
0620 07:06:46.804154 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.804156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.804202 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.804272 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.805590 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.806182 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.807324 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.807593 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.807690 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.810133 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.810136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.810138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.810163 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.811592 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.812119 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.812123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.812125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.812214 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.812575 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.812578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.812579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.812765 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.812843 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.813265 (+ 422us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.813361 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:06:46.813570 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9755
I20260620 07:06:46.813654 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:06:46.813927 8288 external_mini_cluster.cc:1428] Setting key 66810318686525ae893ed2f42fa7ee28
I20260620 07:06:46.814482 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.802250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.3:35339)
0620 07:06:46.802715 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.802774 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.803913 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.804500 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.804508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.804850 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.805437 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.805452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.806290 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.806293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.807214 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.807223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.807845 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.807848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.807986 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.808567 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.808582 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.809983 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.811684 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.811689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.811704 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.812008 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.812305 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.812308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.812310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.812498 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.812846 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.812852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.813080 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.813227 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.813522 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260620 07:06:46.815850 9886 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39607
I20260620 07:06:46.815989 9886 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:46.816566 9886 heartbeater.cc:511] Master 127.8.24.62:39607 requested a full tablet report, sending...
I20260620 07:06:46.817602 9403 ts_manager.cc:194] Registered new tserver with Master: dea59aeb8011487995e80c00507eb92a (127.8.24.3:38755)
I20260620 07:06:46.818313 9403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:35339
I20260620 07:06:46.823904 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.817963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:34723 (local address 127.8.24.62:39607)
0620 07:06:46.818074 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.818078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.818215 (+ 137us) server_negotiation.cc:407] Connection header received
0620 07:06:46.818297 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.818300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.818343 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.818425 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.818896 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.819385 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.820175 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.820379 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.820471 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.821418 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.821421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.821423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.821444 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.822436 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.822799 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.822803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.822804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.822879 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.823109 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.823113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.823115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.823277 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.823357 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.823644 (+ 287us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.823747 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:06:46.823927 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.817896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.8.24.3:34723)
0620 07:06:46.818139 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.818151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.818235 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.818453 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.818456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.818676 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:46.818788 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.818796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.819515 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.819518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.820076 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.820083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.820478 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.820482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.820564 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.820987 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:46.820997 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:46.821290 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:46.822530 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.822533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.822535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.822718 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.822957 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:46.822960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:46.822962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:46.823037 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:46.823359 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:46.823362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:46.823476 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.823581 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.823735 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260620 07:06:46.827543 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 07:06:46 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:46.851517 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.843792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.8.24.62:39607)
0620 07:06:46.843906 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.843909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.843921 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:06:46.843964 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.843968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.844008 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.844087 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:46.844681 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.845307 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.846339 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.846544 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.846619 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.848725 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:46.848728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:46.848730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:46.848755 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:46.849839 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.850338 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.850341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.850342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.850439 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:46.850729 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:46.850733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:46.850735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:46.850957 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:46.851065 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.851254 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.851348 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:06:46.854326 9403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:06:46.857447 9403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:46.871428 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.865519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.8.24.1:46823)
0620 07:06:46.866587 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.866600 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.866621 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:06:46.866693 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.866698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.866861 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.867011 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:46.867253 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868387 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.869998 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.870800 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.870910 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871066 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.871216 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.871302 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":68,"threads_started":1}
I20260620 07:06:46.871505 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.865383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46823 (local address 127.0.0.1:49136)
0620 07:06:46.866162 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.866213 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.866339 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.867003 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.867010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.867030 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:46.867153 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.867159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868492 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.868495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.869893 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.869904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.870917 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.870921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.871031 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871073 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.871191 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.871327 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":74,"spinlock_wait_cycles":5888,"thread_start_us":81,"threads_started":1}
I20260620 07:06:46.871835 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.865934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45825 (local address 127.0.0.1:51262)
0620 07:06:46.866545 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.866556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.866624 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.867210 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.867213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.867229 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:46.867428 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.867468 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868381 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.868384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.870383 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.870391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.871478 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.871481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.871562 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871585 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.871654 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.871720 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":1043,"thread_start_us":82,"threads_started":1}
I20260620 07:06:46.872205 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.865472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38755 (local address 127.0.0.1:55484)
0620 07:06:46.866170 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.866212 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.866339 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.867220 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.867223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.867235 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:46.867426 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.867457 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868691 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.868694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.869842 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.869851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.871836 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.871840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.871934 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871956 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.872016 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.872080 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":79,"spinlock_wait_cycles":4352,"thread_start_us":100,"threads_started":1}
I20260620 07:06:46.872268 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.866177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.8.24.2:45825)
0620 07:06:46.866570 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.866582 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.866606 (+ 24us) server_negotiation.cc:407] Connection header received
0620 07:06:46.866737 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.866742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.866933 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.867079 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:46.867576 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868267 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.870499 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.871156 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.871243 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871389 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.872047 (+ 658us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.872140 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":98,"threads_started":1}
I20260620 07:06:46.872594 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.865750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.8.24.3:38755)
0620 07:06:46.866777 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.866786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.866803 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:06:46.866860 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.866864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.867011 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.867118 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:46.867574 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.868607 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.869949 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.870786 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.870895 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.871066 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.872371 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.872465 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":91,"threads_started":1}
I20260620 07:06:46.874765 9680 tablet_service.cc:1467] Processing CreateTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 (DEFAULT_TABLE table=test-table [id=73058a73628446c59bccc1ebd8eb3185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.874662 9540 tablet_service.cc:1467] Processing CreateTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 (DEFAULT_TABLE table=test-table [id=73058a73628446c59bccc1ebd8eb3185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.874940 9820 tablet_service.cc:1467] Processing CreateTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 (DEFAULT_TABLE table=test-table [id=73058a73628446c59bccc1ebd8eb3185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.875800 9820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9fcf615acb74c7e874d3176f3ab6f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.875798 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9fcf615acb74c7e874d3176f3ab6f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.875798 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9fcf615acb74c7e874d3176f3ab6f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.884044 9910 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Bootstrap starting.
I20260620 07:06:46.884809 9909 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Bootstrap starting.
I20260620 07:06:46.885950 9910 tablet_bootstrap.cc:654] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.886437 9909 tablet_bootstrap.cc:654] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.886648 9910 log.cc:824] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:46.887084 9909 log.cc:824] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:46.887645 9911 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Bootstrap starting.
I20260620 07:06:46.888638 9910 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: No bootstrap required, opened a new log
I20260620 07:06:46.888814 9909 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: No bootstrap required, opened a new log
I20260620 07:06:46.888824 9910 ts_tablet_manager.cc:1397] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:06:46.888986 9909 ts_tablet_manager.cc:1397] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 07:06:46.889443 9911 tablet_bootstrap.cc:654] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.890192 9911 log.cc:824] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:46.891747 9911 tablet_bootstrap.cc:492] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: No bootstrap required, opened a new log
I20260620 07:06:46.891916 9911 ts_tablet_manager.cc:1397] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260620 07:06:46.892688 9910 raft_consensus.cc:348] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.892693 9909 raft_consensus.cc:348] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.892957 9910 raft_consensus.cc:374] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.892957 9909 raft_consensus.cc:374] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.893013 9910 raft_consensus.cc:729] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13f7215bfc74c96be9883b0c1b48ce1, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.893013 9909 raft_consensus.cc:729] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea59aeb8011487995e80c00507eb92a, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.893453 9910 consensus_queue.cc:260] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.893452 9909 consensus_queue.cc:260] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.894362 9886 heartbeater.cc:503] Master 127.8.24.62:39607 was elected leader, sending a full tablet report...
I20260620 07:06:46.894486 9910 ts_tablet_manager.cc:1428] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 07:06:46.894487 9909 ts_tablet_manager.cc:1428] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:06:46.895303 9911 raft_consensus.cc:348] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.895524 9606 heartbeater.cc:503] Master 127.8.24.62:39607 was elected leader, sending a full tablet report...
I20260620 07:06:46.895571 9911 raft_consensus.cc:374] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.895622 9911 raft_consensus.cc:729] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748016b42c0d4fa6ab946b87c0a62bf2, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.896019 9911 consensus_queue.cc:260] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.896869 9746 heartbeater.cc:503] Master 127.8.24.62:39607 was elected leader, sending a full tablet report...
I20260620 07:06:46.897406 9911 ts_tablet_manager.cc:1428] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 07:06:46.901989 9607 tablet.cc:2369] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:46.909621 9916 raft_consensus.cc:482] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:46.909788 9916 raft_consensus.cc:504] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.910857 9916 leader_election.cc:288] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823), 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825)
I20260620 07:06:46.915074 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.911143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45825 (local address 127.8.24.3:33587)
0620 07:06:46.911330 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.911347 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.911456 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.911682 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.911685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.911698 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:46.911808 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.911813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.912774 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.912777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.914035 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.914045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.914666 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.914768 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.914790 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.914854 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.914927 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260620 07:06:46.915062 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.911226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:33587 (local address 127.8.24.2:45825)
0620 07:06:46.911399 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.911404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.911419 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:06:46.911551 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.911553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.911595 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.911660 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:46.911885 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.912687 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914158 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.914560 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914652 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.914715 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.914881 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.914957 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260620 07:06:46.915302 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.911270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:34907 (local address 127.8.24.1:46823)
0620 07:06:46.911384 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.911388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.911672 (+ 284us) server_negotiation.cc:407] Connection header received
0620 07:06:46.911760 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.911763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.911825 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.911903 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:46.912141 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.912828 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914448 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.914828 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914924 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.914981 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.915135 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.915204 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 07:06:46.915313 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.911143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46823 (local address 127.8.24.3:34907)
0620 07:06:46.911588 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:46.911600 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:46.911685 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:46.911909 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:46.911911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:46.911925 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:46.912011 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.912014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.912908 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.912911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.914343 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:46.914350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.914930 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.914933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:46.915027 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.915047 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 07:06:46.915108 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:06:46.915170 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":42,"thread_start_us":116,"threads_started":1}
I20260620 07:06:46.915956 9560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9fcf615acb74c7e874d3176f3ab6f10" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" is_pre_election: true
I20260620 07:06:46.915969 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9fcf615acb74c7e874d3176f3ab6f10" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" is_pre_election: true
I20260620 07:06:46.916262 9560 raft_consensus.cc:2454] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 0.
I20260620 07:06:46.916261 9700 raft_consensus.cc:2454] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 0.
I20260620 07:06:46.916770 9773 leader_election.cc:302] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 748016b42c0d4fa6ab946b87c0a62bf2, a13f7215bfc74c96be9883b0c1b48ce1, dea59aeb8011487995e80c00507eb92a; no voters:
I20260620 07:06:46.917045 9916 raft_consensus.cc:2781] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:46.917119 9916 raft_consensus.cc:482] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:46.917150 9916 raft_consensus.cc:3037] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:46.918143 9916 raft_consensus.cc:504] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.918464 9916 leader_election.cc:288] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 election: Requested vote from peers a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823), 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825)
I20260620 07:06:46.918759 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9fcf615acb74c7e874d3176f3ab6f10" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
I20260620 07:06:46.918759 9560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9fcf615acb74c7e874d3176f3ab6f10" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f7215bfc74c96be9883b0c1b48ce1"
I20260620 07:06:46.918870 9560 raft_consensus.cc:3037] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:46.918871 9700 raft_consensus.cc:3037] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:46.920095 9560 raft_consensus.cc:2454] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 1.
I20260620 07:06:46.920423 9773 leader_election.cc:302] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13f7215bfc74c96be9883b0c1b48ce1, dea59aeb8011487995e80c00507eb92a; no voters:
I20260620 07:06:46.920640 9916 raft_consensus.cc:2781] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:46.920814 9916 raft_consensus.cc:686] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 1 LEADER]: Becoming Leader. State: Replica: dea59aeb8011487995e80c00507eb92a, State: Running, Role: LEADER
I20260620 07:06:46.921039 9700 raft_consensus.cc:2454] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 1.
I20260620 07:06:46.921166 9916 consensus_queue.cc:237] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } }
I20260620 07:06:46.924192 9403 catalog_manager.cc:5495] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a reported cstate change: term changed from 0 to 1, leader changed from <none> to dea59aeb8011487995e80c00507eb92a (127.8.24.3). New cstate: current_term: 1 leader_uuid: "dea59aeb8011487995e80c00507eb92a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:46.941344 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:46.937960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.8.24.3:38755)
0620 07:06:46.938063 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:46.938066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:46.938078 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:06:46.938253 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:46.938255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:46.938308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:46.938386 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:46.938592 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.939084 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.939940 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:46.940209 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:46.940279 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:46.940582 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:46.940998 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:46.941091 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:46.941180 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 07:06:46.941233 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 07:06:46.946915 9560 raft_consensus.cc:1261] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Refusing update from remote peer dea59aeb8011487995e80c00507eb92a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:46.946913 9700 raft_consensus.cc:1261] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Refusing update from remote peer dea59aeb8011487995e80c00507eb92a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:46.947558 9919 consensus_queue.cc:1035] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:46.947742 9916 consensus_queue.cc:1035] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:46.954493 9922 mvcc.cc:205] Tried to move back new op lower bound from 7298822991648104448 to 7298822991551672320. Current Snapshot: MvccSnapshot[applied={T|T < 7298822991648104448}]
I20260620 07:06:46.956668 9924 mvcc.cc:205] Tried to move back new op lower bound from 7298822991648104448 to 7298822991551672320. Current Snapshot: MvccSnapshot[applied={T|T < 7298822991648104448}]
I20260620 07:06:46.958560 9923 mvcc.cc:205] Tried to move back new op lower bound from 7298822991648104448 to 7298822991551672320. Current Snapshot: MvccSnapshot[applied={T|T < 7298822991648104448}]
I20260620 07:06:46.965381 9402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:46.965582 9402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:46.968067 9402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Sending DeleteTablet for 3 replicas of tablet d9fcf615acb74c7e874d3176f3ab6f10
I20260620 07:06:46.968843 9820 tablet_service.cc:1514] Processing DeleteTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:46 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55484
I20260620 07:06:46.968859 9680 tablet_service.cc:1514] Processing DeleteTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:46 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51262
I20260620 07:06:46.968837 9540 tablet_service.cc:1514] Processing DeleteTablet for tablet d9fcf615acb74c7e874d3176f3ab6f10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:46 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49136
I20260620 07:06:46.969345 9934 tablet_replica.cc:330] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: stopping tablet replica
I20260620 07:06:46.969527 9932 tablet_replica.cc:330] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: stopping tablet replica
I20260620 07:06:46.969672 9934 raft_consensus.cc:2229] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:46.969899 9932 raft_consensus.cc:2229] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:46.970002 9934 raft_consensus.cc:2258] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:46.970220 9402 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 07:06:46.970336 9932 raft_consensus.cc:2258] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:06:46.970659 9402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:46.971544 9934 ts_tablet_manager.cc:1905] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:46.971985 9933 tablet_replica.cc:330] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: stopping tablet replica
I20260620 07:06:46.972136 9932 ts_tablet_manager.cc:1905] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:46.972283 9933 raft_consensus.cc:2229] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:46.972566 9933 raft_consensus.cc:2258] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:46.975337 9932 ts_tablet_manager.cc:1918] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:06:46.975517 9932 log.cc:1196] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/d9fcf615acb74c7e874d3176f3ab6f10
I20260620 07:06:46.975883 9932 ts_tablet_manager.cc:1939] T d9fcf615acb74c7e874d3176f3ab6f10 P dea59aeb8011487995e80c00507eb92a: Deleting consensus metadata
I20260620 07:06:46.975988 9933 ts_tablet_manager.cc:1905] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:46.976627 9540 tablet_service.cc:1467] Processing CreateTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 (DEFAULT_TABLE table=test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.976873 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e56887a0ab4bc8ace14c73799638b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.978489 9933 ts_tablet_manager.cc:1918] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:06:46.978572 9933 log.cc:1196] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/d9fcf615acb74c7e874d3176f3ab6f10
I20260620 07:06:46.978948 9933 ts_tablet_manager.cc:1939] T d9fcf615acb74c7e874d3176f3ab6f10 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting consensus metadata
I20260620 07:06:46.979563 9910 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Bootstrap starting.
I20260620 07:06:46.979732 9680 tablet_service.cc:1467] Processing CreateTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 (DEFAULT_TABLE table=test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.979960 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e56887a0ab4bc8ace14c73799638b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.980412 9910 tablet_bootstrap.cc:654] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.981698 9910 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: No bootstrap required, opened a new log
I20260620 07:06:46.981775 9910 ts_tablet_manager.cc:1397] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:46.981968 9911 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Bootstrap starting.
I20260620 07:06:46.982183 9910 raft_consensus.cc:348] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.982277 9910 raft_consensus.cc:374] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.982306 9910 raft_consensus.cc:729] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13f7215bfc74c96be9883b0c1b48ce1, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.982417 9910 consensus_queue.cc:260] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.982631 9910 ts_tablet_manager.cc:1428] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:46.982820 9911 tablet_bootstrap.cc:654] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.983529 9387 catalog_manager.cc:4841] TS 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825): tablet d9fcf615acb74c7e874d3176f3ab6f10 (table test-table [id=73058a73628446c59bccc1ebd8eb3185]) successfully deleted
I20260620 07:06:46.983851 9388 catalog_manager.cc:4841] TS dea59aeb8011487995e80c00507eb92a (127.8.24.3:38755): tablet d9fcf615acb74c7e874d3176f3ab6f10 (table test-table [id=73058a73628446c59bccc1ebd8eb3185]) successfully deleted
I20260620 07:06:46.984169 9820 tablet_service.cc:1467] Processing CreateTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 (DEFAULT_TABLE table=test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:46.984429 9820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e56887a0ab4bc8ace14c73799638b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:46.984858 9934 ts_tablet_manager.cc:1918] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:06:46.984943 9934 log.cc:1196] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/d9fcf615acb74c7e874d3176f3ab6f10
I20260620 07:06:46.985201 9934 ts_tablet_manager.cc:1939] T d9fcf615acb74c7e874d3176f3ab6f10 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting consensus metadata
I20260620 07:06:46.986722 9909 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Bootstrap starting.
I20260620 07:06:46.986920 9389 catalog_manager.cc:4841] TS a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823): tablet d9fcf615acb74c7e874d3176f3ab6f10 (table test-table [id=73058a73628446c59bccc1ebd8eb3185]) successfully deleted
I20260620 07:06:46.987625 9909 tablet_bootstrap.cc:654] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:46.988687 9909 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: No bootstrap required, opened a new log
I20260620 07:06:46.988780 9909 ts_tablet_manager.cc:1397] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:46.989281 9909 raft_consensus.cc:348] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.989349 9911 tablet_bootstrap.cc:492] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: No bootstrap required, opened a new log
I20260620 07:06:46.989379 9909 raft_consensus.cc:374] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.989410 9909 raft_consensus.cc:729] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea59aeb8011487995e80c00507eb92a, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.989424 9911 ts_tablet_manager.cc:1397] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260620 07:06:46.989521 9909 consensus_queue.cc:260] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.989758 9909 ts_tablet_manager.cc:1428] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:46.989825 9911 raft_consensus.cc:348] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.989941 9911 raft_consensus.cc:374] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:46.989976 9911 raft_consensus.cc:729] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748016b42c0d4fa6ab946b87c0a62bf2, State: Initialized, Role: FOLLOWER
I20260620 07:06:46.990088 9911 consensus_queue.cc:260] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:46.990314 9911 ts_tablet_manager.cc:1428] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260620 07:06:47.062480 9887 tablet.cc:2369] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:06:47.109185 9747 tablet.cc:2369] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:47.266050 9928 raft_consensus.cc:482] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:47.266207 9928 raft_consensus.cc:504] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.266624 9928 leader_election.cc:288] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823), 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825)
I20260620 07:06:47.267120 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e56887a0ab4bc8ace14c73799638b6" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" is_pre_election: true
I20260620 07:06:47.267136 9560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e56887a0ab4bc8ace14c73799638b6" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" is_pre_election: true
I20260620 07:06:47.267266 9560 raft_consensus.cc:2454] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 0.
I20260620 07:06:47.267283 9700 raft_consensus.cc:2454] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 0.
I20260620 07:06:47.267597 9773 leader_election.cc:302] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13f7215bfc74c96be9883b0c1b48ce1, dea59aeb8011487995e80c00507eb92a; no voters:
I20260620 07:06:47.267764 9928 raft_consensus.cc:2781] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:47.267832 9928 raft_consensus.cc:482] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:47.267874 9928 raft_consensus.cc:3037] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.268818 9928 raft_consensus.cc:504] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.269109 9928 leader_election.cc:288] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 election: Requested vote from peers a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823), 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825)
I20260620 07:06:47.269472 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e56887a0ab4bc8ace14c73799638b6" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
I20260620 07:06:47.269505 9560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e56887a0ab4bc8ace14c73799638b6" candidate_uuid: "dea59aeb8011487995e80c00507eb92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f7215bfc74c96be9883b0c1b48ce1"
I20260620 07:06:47.269584 9700 raft_consensus.cc:3037] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.269610 9560 raft_consensus.cc:3037] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.270466 9700 raft_consensus.cc:2454] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 1.
I20260620 07:06:47.270524 9560 raft_consensus.cc:2454] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea59aeb8011487995e80c00507eb92a in term 1.
I20260620 07:06:47.270722 9771 leader_election.cc:302] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 748016b42c0d4fa6ab946b87c0a62bf2, dea59aeb8011487995e80c00507eb92a; no voters:
I20260620 07:06:47.270864 9928 raft_consensus.cc:2781] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:47.270949 9928 raft_consensus.cc:686] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 1 LEADER]: Becoming Leader. State: Replica: dea59aeb8011487995e80c00507eb92a, State: Running, Role: LEADER
I20260620 07:06:47.271087 9928 consensus_queue.cc:237] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.272660 9402 catalog_manager.cc:5495] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a reported cstate change: term changed from 0 to 1, leader changed from <none> to dea59aeb8011487995e80c00507eb92a (127.8.24.3). New cstate: current_term: 1 leader_uuid: "dea59aeb8011487995e80c00507eb92a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } health_report { overall_health: HEALTHY } } }
Jun 20 07:06:47 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:47.316852 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.309136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37888 (local address 127.8.24.62:39607)
0620 07:06:47.309241 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.309245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.309476 (+ 231us) server_negotiation.cc:407] Connection header received
0620 07:06:47.309595 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.309597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.309633 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.309694 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:47.310341 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.310844 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.311662 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.311903 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.311987 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.314188 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:47.314201 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:47.314205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:47.314226 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:47.315326 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.315752 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.315756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.315758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.315824 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.316041 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.316044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.316046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.316213 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:47.316290 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.316566 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.316715 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":34}
I20260620 07:06:47.316852 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.309089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39607 (local address 127.0.0.1:37888)
0620 07:06:47.309385 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.309408 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.309518 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.309723 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.309726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.310138 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:47.310233 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.310241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.310940 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.310943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.311556 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.311566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.311995 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.311998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.312072 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.312613 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:47.312625 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:47.314061 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:47.315425 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:47.315453 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:47.315460 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:47.315671 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:47.315896 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:47.315899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:47.315901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:47.315971 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:47.316300 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:47.316307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:47.316422 (+ 115us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.316513 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.316664 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":120,"threads_started":1}
I20260620 07:06:47.320745 9402 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:37888:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 07:06:47.321404 9402 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:47.325846 9540 tablet_service.cc:1467] Processing CreateTablet for tablet edf073bd96654efc818e93dabdd2a4ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9e8c1dd4e8b4b98a80c873b48e0098d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:47.325875 9680 tablet_service.cc:1467] Processing CreateTablet for tablet edf073bd96654efc818e93dabdd2a4ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9e8c1dd4e8b4b98a80c873b48e0098d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:47.325942 9820 tablet_service.cc:1467] Processing CreateTablet for tablet edf073bd96654efc818e93dabdd2a4ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9e8c1dd4e8b4b98a80c873b48e0098d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:47.326258 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edf073bd96654efc818e93dabdd2a4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:47.326256 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edf073bd96654efc818e93dabdd2a4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:47.326256 9820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edf073bd96654efc818e93dabdd2a4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:47.328856 9910 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: Bootstrap starting.
I20260620 07:06:47.329131 9909 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: Bootstrap starting.
I20260620 07:06:47.329131 9911 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: Bootstrap starting.
I20260620 07:06:47.329710 9910 tablet_bootstrap.cc:654] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:47.330001 9909 tablet_bootstrap.cc:654] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:47.330001 9911 tablet_bootstrap.cc:654] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:47.331068 9911 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: No bootstrap required, opened a new log
I20260620 07:06:47.331068 9909 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: No bootstrap required, opened a new log
I20260620 07:06:47.331151 9909 ts_tablet_manager.cc:1397] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:47.331151 9911 ts_tablet_manager.cc:1397] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:47.331578 9911 raft_consensus.cc:348] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.331578 9909 raft_consensus.cc:348] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.331694 9911 raft_consensus.cc:374] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:47.331694 9909 raft_consensus.cc:374] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:47.331745 9909 raft_consensus.cc:729] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea59aeb8011487995e80c00507eb92a, State: Initialized, Role: FOLLOWER
I20260620 07:06:47.331745 9911 raft_consensus.cc:729] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748016b42c0d4fa6ab946b87c0a62bf2, State: Initialized, Role: FOLLOWER
I20260620 07:06:47.331869 9911 consensus_queue.cc:260] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.331869 9909 consensus_queue.cc:260] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.332129 9909 ts_tablet_manager.cc:1428] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:47.332110 9917 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.332270 9917 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.332439 9928 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.332551 9928 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.332783 9910 tablet_bootstrap.cc:492] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: No bootstrap required, opened a new log
I20260620 07:06:47.332873 9910 ts_tablet_manager.cc:1397] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 07:06:47.333235 9910 raft_consensus.cc:348] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.333326 9910 raft_consensus.cc:374] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:47.333352 9910 raft_consensus.cc:729] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13f7215bfc74c96be9883b0c1b48ce1, State: Initialized, Role: FOLLOWER
I20260620 07:06:47.333477 9910 consensus_queue.cc:260] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.333734 9910 ts_tablet_manager.cc:1428] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:47.333776 9915 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.333885 9915 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.332129 9911 ts_tablet_manager.cc:1428] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:47.338228 9915 raft_consensus.cc:482] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:47.338311 9915 raft_consensus.cc:504] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.339265 9915 leader_election.cc:288] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825), dea59aeb8011487995e80c00507eb92a (127.8.24.3:38755)
I20260620 07:06:47.343575 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.339621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56725 (local address 127.8.24.3:38755)
0620 07:06:47.339830 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.339834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.340277 (+ 443us) server_negotiation.cc:407] Connection header received
0620 07:06:47.340387 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.340390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.340435 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.340502 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:47.340735 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.341420 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.342523 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.342977 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.343077 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.343129 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.343354 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.343455 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260620 07:06:47.343616 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.339598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41337 (local address 127.8.24.2:45825)
0620 07:06:47.339741 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.339745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.339912 (+ 167us) server_negotiation.cc:407] Connection header received
0620 07:06:47.340066 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.340068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.340114 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.340186 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:47.340482 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.341218 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.342557 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.342970 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.343083 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.343153 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.343417 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.343522 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 07:06:47.343611 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.339543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45825 (local address 127.8.24.1:41337)
0620 07:06:47.339829 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.339843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.339946 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.340206 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.340208 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.340227 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:47.340376 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.340383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.341318 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.341321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.342389 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.342405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.343207 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.343210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.343312 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.343331 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.343388 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.343459 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":37,"spinlock_wait_cycles":10496,"thread_start_us":127,"threads_started":1}
I20260620 07:06:47.343609 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.339552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38755 (local address 127.8.24.1:56725)
0620 07:06:47.340187 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.340199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.340300 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.340521 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.340523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.340540 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:47.340642 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.340648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.341508 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.341512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.342389 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.342399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.343092 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.343095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.343198 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.343251 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.343317 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.343386 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":58,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260620 07:06:47.344067 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf073bd96654efc818e93dabdd2a4ed" candidate_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" is_pre_election: true
I20260620 07:06:47.344209 9700 raft_consensus.cc:2454] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a13f7215bfc74c96be9883b0c1b48ce1 in term 0.
I20260620 07:06:47.344375 9840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf073bd96654efc818e93dabdd2a4ed" candidate_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea59aeb8011487995e80c00507eb92a" is_pre_election: true
I20260620 07:06:47.344532 9491 leader_election.cc:302] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 748016b42c0d4fa6ab946b87c0a62bf2, a13f7215bfc74c96be9883b0c1b48ce1; no voters:
I20260620 07:06:47.344599 9840 raft_consensus.cc:2454] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a13f7215bfc74c96be9883b0c1b48ce1 in term 0.
I20260620 07:06:47.344764 9915 raft_consensus.cc:2781] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:47.344823 9915 raft_consensus.cc:482] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:47.344866 9915 raft_consensus.cc:3037] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.345880 9915 raft_consensus.cc:504] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.346203 9915 leader_election.cc:288] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [CANDIDATE]: Term 1 election: Requested vote from peers 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825), dea59aeb8011487995e80c00507eb92a (127.8.24.3:38755)
I20260620 07:06:47.346487 9700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf073bd96654efc818e93dabdd2a4ed" candidate_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "748016b42c0d4fa6ab946b87c0a62bf2"
I20260620 07:06:47.346514 9840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf073bd96654efc818e93dabdd2a4ed" candidate_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea59aeb8011487995e80c00507eb92a"
I20260620 07:06:47.346601 9700 raft_consensus.cc:3037] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.346601 9840 raft_consensus.cc:3037] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:47.347728 9700 raft_consensus.cc:2454] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a13f7215bfc74c96be9883b0c1b48ce1 in term 1.
I20260620 07:06:47.347728 9840 raft_consensus.cc:2454] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a13f7215bfc74c96be9883b0c1b48ce1 in term 1.
I20260620 07:06:47.347980 9492 leader_election.cc:302] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13f7215bfc74c96be9883b0c1b48ce1, dea59aeb8011487995e80c00507eb92a; no voters:
I20260620 07:06:47.348165 9915 raft_consensus.cc:2781] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:47.348335 9915 raft_consensus.cc:686] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 LEADER]: Becoming Leader. State: Replica: a13f7215bfc74c96be9883b0c1b48ce1, State: Running, Role: LEADER
I20260620 07:06:47.348562 9915 consensus_queue.cc:237] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } }
I20260620 07:06:47.349599 9950 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: TxnStatusTablet state changed. Reason: New leader a13f7215bfc74c96be9883b0c1b48ce1. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.349679 9950 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:47.350178 9952 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 07:06:47.350303 9952 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 07:06:47.351132 9397 catalog_manager.cc:5495] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:47.366971 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.363501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49142 (local address 127.8.24.1:46823)
0620 07:06:47.363629 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.363634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.363646 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:06:47.363835 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.363839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.363897 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.363976 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:47.364216 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.364722 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.365567 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.365785 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.365870 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.366149 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.366593 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.366693 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.366774 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.366828 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:06:47.366969 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.363392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46823 (local address 127.0.0.1:49142)
0620 07:06:47.363568 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.363583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.363692 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.364004 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.364007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.364017 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:47.364120 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.364126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.364830 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.364834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.365459 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.365468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.365882 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.365885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.365976 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.366020 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.366686 (+ 666us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.366702 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.366769 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.366858 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 07:06:47.369684 9700 raft_consensus.cc:1261] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Refusing update from remote peer a13f7215bfc74c96be9883b0c1b48ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:47.369845 9840 raft_consensus.cc:1261] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Refusing update from remote peer a13f7215bfc74c96be9883b0c1b48ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:47.370234 9950 consensus_queue.cc:1035] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:47.370735 9950 consensus_queue.cc:1035] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:47.371982 9917 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: TxnStatusTablet state changed. Reason: New leader a13f7215bfc74c96be9883b0c1b48ce1. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.372092 9917 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.372247 9928 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: TxnStatusTablet state changed. Reason: New leader a13f7215bfc74c96be9883b0c1b48ce1. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.372349 9928 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.373524 9950 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.373629 9950 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:47.375178 9950 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.375290 9950 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P a13f7215bfc74c96be9883b0c1b48ce1: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:47.376116 9917 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.376225 9917 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P 748016b42c0d4fa6ab946b87c0a62bf2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.376330 9928 tablet_replica.cc:439] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 } } peers { permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 } } peers { permanent_uuid: "dea59aeb8011487995e80c00507eb92a" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 38755 } } }
I20260620 07:06:47.376456 9928 tablet_replica.cc:442] T edf073bd96654efc818e93dabdd2a4ed P dea59aeb8011487995e80c00507eb92a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:47.388217 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.384613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46823 (local address 127.8.24.3:56491)
0620 07:06:47.384879 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.384891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.384975 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.385225 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.385228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.385236 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:47.385340 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.385345 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.386446 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.386448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.387056 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.387064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.387610 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.387613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.387676 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.387717 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.387982 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.387985 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.388026 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.388102 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260620 07:06:47.388231 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.384707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56491 (local address 127.8.24.1:46823)
0620 07:06:47.384822 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.384826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.384953 (+ 127us) server_negotiation.cc:407] Connection header received
0620 07:06:47.385042 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.385045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.385092 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.385175 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:47.385520 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.386346 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.387149 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.387513 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.387607 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.387827 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.387913 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.387981 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.388057 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.388105 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:06:47.394292 9560 raft_consensus.cc:1261] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Refusing update from remote peer dea59aeb8011487995e80c00507eb92a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:47.394621 9700 raft_consensus.cc:1261] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Refusing update from remote peer dea59aeb8011487995e80c00507eb92a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:47.394708 9928 consensus_queue.cc:1035] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a13f7215bfc74c96be9883b0c1b48ce1" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:47.394955 9919 consensus_queue.cc:1035] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "748016b42c0d4fa6ab946b87c0a62bf2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:47.424680 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.420385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:49601 (local address 127.8.24.3:38755)
0620 07:06:47.420491 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.420495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.420780 (+ 285us) server_negotiation.cc:407] Connection header received
0620 07:06:47.420893 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.420903 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.420975 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.421047 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:47.421519 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.422301 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.423520 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.423818 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.423932 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.424218 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.424305 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.424369 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.424494 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.424545 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:06:47.424700 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.420305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:38755 (local address 127.8.24.1:49601)
0620 07:06:47.420693 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:47.420707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:47.420808 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:47.421280 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:47.421284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:47.421292 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:47.421408 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.421414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.422430 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.422433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.423392 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:47.423403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.423940 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.423943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:47.424040 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.424094 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:47.424408 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:47.424414 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:06:47.424468 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 07:06:47.424546 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":94,"threads_started":1}
I20260620 07:06:47.448642 9397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:47.448796 9397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:47.450939 9397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89a01c7a1ae747fd9209e23531029763: Sending DeleteTablet for 3 replicas of tablet e9e56887a0ab4bc8ace14c73799638b6
I20260620 07:06:47.451545 9820 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:47 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55484
I20260620 07:06:47.451730 9932 tablet_replica.cc:330] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: stopping tablet replica
I20260620 07:06:47.451853 9932 raft_consensus.cc:2229] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:47.451838 9680 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:47 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51262
I20260620 07:06:47.452026 9933 tablet_replica.cc:330] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: stopping tablet replica
I20260620 07:06:47.452039 9932 raft_consensus.cc:2258] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:47.452124 9933 raft_consensus.cc:2229] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:47.452229 9933 raft_consensus.cc:2258] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:47.452791 9540 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e56887a0ab4bc8ace14c73799638b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:47 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49136
I20260620 07:06:47.453035 9934 tablet_replica.cc:330] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: stopping tablet replica
I20260620 07:06:47.453090 9933 ts_tablet_manager.cc:1905] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:47.453130 9934 raft_consensus.cc:2229] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:47.453226 9934 raft_consensus.cc:2258] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:47.454043 9934 ts_tablet_manager.cc:1905] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:47.455508 9933 ts_tablet_manager.cc:1918] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:47.455586 9933 log.cc:1196] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/e9e56887a0ab4bc8ace14c73799638b6
I20260620 07:06:47.455787 9933 ts_tablet_manager.cc:1939] T e9e56887a0ab4bc8ace14c73799638b6 P 748016b42c0d4fa6ab946b87c0a62bf2: Deleting consensus metadata
I20260620 07:06:47.456470 9387 catalog_manager.cc:4841] TS 748016b42c0d4fa6ab946b87c0a62bf2 (127.8.24.2:45825): tablet e9e56887a0ab4bc8ace14c73799638b6 (table test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]) successfully deleted
I20260620 07:06:47.456702 9934 ts_tablet_manager.cc:1918] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:47.456725 9932 ts_tablet_manager.cc:1905] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:47.456775 9934 log.cc:1196] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/e9e56887a0ab4bc8ace14c73799638b6
I20260620 07:06:47.457011 9934 ts_tablet_manager.cc:1939] T e9e56887a0ab4bc8ace14c73799638b6 P a13f7215bfc74c96be9883b0c1b48ce1: Deleting consensus metadata
I20260620 07:06:47.457654 9389 catalog_manager.cc:4841] TS a13f7215bfc74c96be9883b0c1b48ce1 (127.8.24.1:46823): tablet e9e56887a0ab4bc8ace14c73799638b6 (table test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]) successfully deleted
I20260620 07:06:47.459020 9932 ts_tablet_manager.cc:1918] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:47.459090 9932 log.cc:1196] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/e9e56887a0ab4bc8ace14c73799638b6
I20260620 07:06:47.459309 9932 ts_tablet_manager.cc:1939] T e9e56887a0ab4bc8ace14c73799638b6 P dea59aeb8011487995e80c00507eb92a: Deleting consensus metadata
I20260620 07:06:47.459919 9388 catalog_manager.cc:4841] TS dea59aeb8011487995e80c00507eb92a (127.8.24.3:38755): tablet e9e56887a0ab4bc8ace14c73799638b6 (table test-table [id=685abdab0ac74f4fa1abe3f4332fbbce]) successfully deleted
Jun 20 07:06:47 dist-test-slave-wjw3 krb5kdc[9358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:06:47.468859 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.453428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.8.24.1:46823)
0620 07:06:47.453557 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.453562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.459722 (+ 6160us) server_negotiation.cc:407] Connection header received
0620 07:06:47.459846 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.459849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.459903 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.459982 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:47.460770 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.461288 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.462374 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.462587 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.462655 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.465502 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:47.465512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:47.465525 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:47.465549 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:47.467164 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.467649 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.467654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.467656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.467749 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.467984 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.467985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.467987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.468241 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:47.468412 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.468593 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.468692 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 07:06:47.469375 9600 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49154
I20260620 07:06:47.478724 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:47.471116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.8.24.62:39607)
0620 07:06:47.471251 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:47.471256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:47.471334 (+ 78us) server_negotiation.cc:407] Connection header received
0620 07:06:47.471430 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:47.471464 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:47.471545 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:47.471628 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:47.472216 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.472995 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.473768 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:47.474090 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:47.474217 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:47.475419 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:47.475423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:47.475426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:47.475480 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:47.477005 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.477460 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.477464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.477466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.477565 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:47.477833 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:47.477836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:47.477837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:47.478090 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:47.478220 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:47.478428 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 07:06:47.478558 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260620 07:06:47.479161 9397 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37900
I20260620 07:06:47.480266 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9475
I20260620 07:06:47.486568 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9615
I20260620 07:06:47.491456 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9755
I20260620 07:06:47.496855 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9371
2026-06-20T07:06:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2659 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:47 dist-test-slave-wjw3 krb5kdc[9972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:47 dist-test-slave-wjw3 krb5kdc[9972](info): set up 2 sockets
Jun 20 07:06:47 dist-test-slave-wjw3 krb5kdc[9972](info): commencing operation
krb5kdc: starting...
W20260620 07:06:48.447052 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:48 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:48.557621 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41047
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:41047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:48.663841 9985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:48.664091 9985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:48.664129 9985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:48.668088 9985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:48.668138 9985 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 07:06:48.668154 9985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:48.668171 9985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:48.668192 9985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:48.668208 9985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:48.673280 9985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:41047
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:41047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:48.674240 9985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:48.675196 9985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:48.682924 9990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:48.682958 9993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:48.682981 9991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:48.683121 9985 server_base.cc:1034] running on GCE node
I20260620 07:06:48.683753 9985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:48.684744 9985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:48.685889 9985 hybrid_clock.cc:648] HybridClock initialized: now 1781939208685874 us; error 27 us; skew 500 ppm
Jun 20 07:06:48 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:48.688369 9985 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:48.689127 9985 webserver.cc:460] Webserver started at http://127.8.24.62:40239/ using document root <none> and password file <none>
I20260620 07:06:48.689615 9985 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:48.689668 9985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:48.689826 9985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:48.691751 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "b4d569ec7c2541d9b88e2e63b6d8c456"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "c2f68442660f8b7bc220e74141958eac"
server_key_iv: "9cefd99ae66d3c064a798b5853b1fe4f"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.692245 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "b4d569ec7c2541d9b88e2e63b6d8c456"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "c2f68442660f8b7bc220e74141958eac"
server_key_iv: "9cefd99ae66d3c064a798b5853b1fe4f"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.695663 9985 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:06:48.697780 10000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:48.698880 9985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:48.698971 9985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "b4d569ec7c2541d9b88e2e63b6d8c456"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "c2f68442660f8b7bc220e74141958eac"
server_key_iv: "9cefd99ae66d3c064a798b5853b1fe4f"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.699050 9985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:48.711544 9985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:48.713923 9985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:48.714054 9985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:48.721014 9985 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:41047
I20260620 07:06:48.721025 10062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:41047 every 8 connection(s)
I20260620 07:06:48.721833 9985 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:48.723095 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 9985
I20260620 07:06:48.723188 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:48.723410 8288 external_mini_cluster.cc:1428] Setting key e8dcae684c25a151e80acd6b6bbfa486
I20260620 07:06:48.724635 10063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:48.729964 10063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Bootstrap starting.
Jun 20 07:06:48 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:48.732142 10063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:48.733196 10063 log.cc:824] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:48.734892 10063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: No bootstrap required, opened a new log
I20260620 07:06:48.735957 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:48.724700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.8.24.62:41047)
0620 07:06:48.725093 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:48.725107 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:48.725140 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:06:48.725732 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:48.725748 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:48.726054 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:48.726339 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:48.727056 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.728076 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.728895 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.729340 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.729475 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:48.731764 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:48.731768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:48.731780 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:48.731806 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:48.733363 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.733734 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.733739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.733743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.733845 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.734086 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.734089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.734091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.734457 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:48.734621 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:48.734880 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 07:06:48.735247 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":127,"threads_started":1}
I20260620 07:06:48.737785 10063 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } }
I20260620 07:06:48.738025 10063 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:48.738083 10063 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d569ec7c2541d9b88e2e63b6d8c456, State: Initialized, Role: FOLLOWER
I20260620 07:06:48.738452 10063 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } }
I20260620 07:06:48.738569 10063 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:48.738628 10063 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:48.738729 10063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:48.739542 10063 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } }
I20260620 07:06:48.739843 10063 leader_election.cc:302] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4d569ec7c2541d9b88e2e63b6d8c456; no voters:
I20260620 07:06:48.740110 10063 leader_election.cc:288] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:48.740183 10068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:48.740403 10068 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [term 1 LEADER]: Becoming Leader. State: Replica: b4d569ec7c2541d9b88e2e63b6d8c456, State: Running, Role: LEADER
I20260620 07:06:48.740684 10068 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } }
I20260620 07:06:48.741241 10063 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:48.742169 10069 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } } }
I20260620 07:06:48.742283 10069 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:48.742504 10070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4d569ec7c2541d9b88e2e63b6d8c456. Latest consensus state: current_term: 1 leader_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d569ec7c2541d9b88e2e63b6d8c456" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 41047 } } }
I20260620 07:06:48.742588 10070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:48.743050 10077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:06:48.745625 10077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:48.750597 10077 catalog_manager.cc:1347] Generated new cluster ID: 423d89b089c8450fb4f9d3ff152d6daf
I20260620 07:06:48.750661 10077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:48.769843 10077 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:48.770995 10077 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:48.777223 10077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Generated new TSK 0
I20260620 07:06:48.777839 10077 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:06:48.810055 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41047
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:48.914548 10089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:48.914769 10089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:48.914811 10089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:48.918622 10089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:48.918666 10089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:48.918701 10089 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 07:06:48.918720 10089 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:48.918748 10089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:06:48.923641 10089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:48.924538 10089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:48.925434 10089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:48.933952 10095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:48.933930 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:48.933964 10094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:48.934008 10089 server_base.cc:1034] running on GCE node
I20260620 07:06:48.934621 10089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:48.935137 10089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:48.936311 10089 hybrid_clock.cc:648] HybridClock initialized: now 1781939208936276 us; error 24 us; skew 500 ppm
Jun 20 07:06:48 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:48.938691 10089 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:06:48.939577 10089 webserver.cc:460] Webserver started at http://127.8.24.1:38227/ using document root <none> and password file <none>
I20260620 07:06:48.940074 10089 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:48.940136 10089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:48.940344 10089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:48.942271 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "1750525b7c8eaff434aaedbfb393a98d"
server_key_iv: "0f5f8f5952eab021ec3c742d8fe5f3b0"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.942795 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "1750525b7c8eaff434aaedbfb393a98d"
server_key_iv: "0f5f8f5952eab021ec3c742d8fe5f3b0"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.946211 10089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:06:48.948431 10104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:48.949477 10089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:48.949577 10089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
format_stamp: "Formatted at 2026-06-20 07:06:48 on dist-test-slave-wjw3"
server_key: "1750525b7c8eaff434aaedbfb393a98d"
server_key_iv: "0f5f8f5952eab021ec3c742d8fe5f3b0"
server_key_version: "encryptionkey@0"
I20260620 07:06:48.949666 10089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:48.960435 10089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:48.962787 10089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:48.962935 10089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:48.964215 10089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:48.964262 10089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:48.964308 10089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:48.964337 10089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:06:48 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:48.973860 10089 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:34503
I20260620 07:06:48.973971 10219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:34503 every 8 connection(s)
I20260620 07:06:48.974709 10089 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:06:48.975203 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10089
I20260620 07:06:48.975330 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:06:48.975682 8288 external_mini_cluster.cc:1428] Setting key 3d7a787156a485de1e80c79599b983a7
I20260620 07:06:48.976392 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:48.964820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40281 (local address 127.8.24.62:41047)
0620 07:06:48.964983 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:48.964989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:48.966215 (+ 1226us) server_negotiation.cc:407] Connection header received
0620 07:06:48.967136 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:48.967141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:48.967195 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:48.967283 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:48.968444 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.969003 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.970080 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.970314 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.970402 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:48.972862 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:48.972866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:48.972868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:48.972897 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:48.974298 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.974798 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.974801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.974803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.974904 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.975281 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.975285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.975287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.975527 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:48.975619 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:48.976111 (+ 492us) server_negotiation.cc:299] Negotiation successful
0620 07:06:48.976235 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 07:06:48.977216 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:48.965578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.1:40281)
0620 07:06:48.966074 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:48.966118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:48.966935 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:48.967493 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:48.967502 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:48.967830 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:48.968308 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:48.968321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.969110 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.969113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:48.969966 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:48.969976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.970462 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.970466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:48.970605 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:48.971112 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:48.971126 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:48.972711 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:48.974396 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:48.974401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:48.974414 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:48.974693 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:48.974997 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:48.975000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:48.975002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:48.975167 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:48.975636 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:48.975642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:48.975891 (+ 249us) client_negotiation.cc:769] Sending connection context
0620 07:06:48.976052 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 07:06:48.976336 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":107,"threads_started":1}
I20260620 07:06:48.979280 10220 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41047
I20260620 07:06:48.979408 10220 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:48.979863 10220 heartbeater.cc:511] Master 127.8.24.62:41047 requested a full tablet report, sending...
I20260620 07:06:48.981288 10017 ts_manager.cc:194] Registered new tserver with Master: 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503)
I20260620 07:06:48.982749 10017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:40281
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:06:48.989161 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:48.983355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:37103 (local address 127.8.24.62:41047)
0620 07:06:48.983485 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:48.983488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:48.983502 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:06:48.983531 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:48.983534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:48.983572 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:48.983642 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:48.984031 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.984514 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.985243 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.985454 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.985534 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:48.986570 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:48.986573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:48.986575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:48.986595 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:48.987792 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.988175 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.988177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.988178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.988244 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:48.988496 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:48.988498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:48.988499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:48.988635 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:48.988700 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:48.988926 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 07:06:48.989019 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:06:48.989177 10227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:48.981123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.1:37103)
0620 07:06:48.981360 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:48.981373 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:48.981460 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:48.983655 (+ 2195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:48.983658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:48.983827 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:48.983934 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:48.983940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.984623 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.984626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:48.985145 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:48.985152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:48.985545 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:48.985549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:48.985630 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:48.986040 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:48.986050 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:48.986330 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:48.987880 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:48.987883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:48.987885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:48.988091 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:48.988319 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:48.988321 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:48.988323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:48.988411 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:48.988705 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:48.988708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:48.988790 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 07:06:48.988883 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 07:06:48.989002 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:06:49.000319 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41047
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:49.106822 10229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:49.107059 10229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:49.107102 10229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:49.110970 10229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:49.111019 10229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:49.111057 10229 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 07:06:49.111075 10229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:49.111106 10229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:06:49.116153 10229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:49.117053 10229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:49.117941 10229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:49.126806 10235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:49.126854 10234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:49.126876 10237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:49.126941 10229 server_base.cc:1034] running on GCE node
I20260620 07:06:49.127532 10229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:49.128108 10229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:49.129264 10229 hybrid_clock.cc:648] HybridClock initialized: now 1781939209129238 us; error 42 us; skew 500 ppm
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:49.131935 10229 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:06:49.132783 10229 webserver.cc:460] Webserver started at http://127.8.24.2:44677/ using document root <none> and password file <none>
I20260620 07:06:49.133296 10229 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:49.133358 10229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:49.133533 10229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:49.135546 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "7e45032335974805b36fc5c34a6393c6"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "9a8d795d57c1df098e570171fcce6ee8"
server_key_iv: "9042826932d26784af8c7bba51095b39"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.136072 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "7e45032335974805b36fc5c34a6393c6"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "9a8d795d57c1df098e570171fcce6ee8"
server_key_iv: "9042826932d26784af8c7bba51095b39"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.139539 10229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:06:49.141872 10244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:49.143051 10229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:49.143147 10229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "7e45032335974805b36fc5c34a6393c6"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "9a8d795d57c1df098e570171fcce6ee8"
server_key_iv: "9042826932d26784af8c7bba51095b39"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.143230 10229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:49.157227 10229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:49.159830 10229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:49.159989 10229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:49.161284 10229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:49.161334 10229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:49.161381 10229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:49.161410 10229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:49.170780 10229 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43601
I20260620 07:06:49.170835 10359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43601 every 8 connection(s)
I20260620 07:06:49.171629 10229 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:06:49.173656 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.162072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:42653 (local address 127.8.24.62:41047)
0620 07:06:49.162182 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.162186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.163165 (+ 979us) server_negotiation.cc:407] Connection header received
0620 07:06:49.164127 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.164131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.164185 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.164253 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.165810 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.166610 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.168010 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.168319 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.168406 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.170603 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.170607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.170609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.170636 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.172082 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.172554 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.172557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.172559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.172634 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.172863 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.172865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.172866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.173014 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.173080 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.173453 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.173544 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:06:49.174342 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.162599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.2:42653)
0620 07:06:49.163031 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.163070 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.163885 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.164628 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.164637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.165038 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:49.165648 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.165667 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.166736 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.166740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.167898 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.167908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.168465 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.168470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.168620 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.169117 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:49.169135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:49.170423 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:49.172198 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.172204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.172217 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.172480 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.172697 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.172699 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.172701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.172812 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.173080 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:49.173084 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:49.173281 (+ 197us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.173418 (+ 137us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.173631 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260620 07:06:49.175469 10360 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41047
I20260620 07:06:49.175607 10360 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:49.175668 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10229
I20260620 07:06:49.175787 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:06:49.176074 8288 external_mini_cluster.cc:1428] Setting key b0a753777debf523a47d2b5bd6e444c2
I20260620 07:06:49.176116 10360 heartbeater.cc:511] Master 127.8.24.62:41047 requested a full tablet report, sending...
I20260620 07:06:49.177047 10017 ts_manager.cc:194] Registered new tserver with Master: 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601)
I20260620 07:06:49.177712 10017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:42653
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:06:49.186755 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.177500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47455 (local address 127.8.24.62:41047)
0620 07:06:49.177592 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.177596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.179172 (+ 1576us) server_negotiation.cc:407] Connection header received
0620 07:06:49.179260 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.179263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.179306 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.179402 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.179973 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.180718 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.181767 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.182094 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.182182 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.183339 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.183343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.183345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.183371 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.184964 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.185451 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.185455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.185457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.185535 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.185831 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.185834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.185835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.186113 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.186198 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.186517 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.186606 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:06:49.186774 10366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.177466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.2:47455)
0620 07:06:49.179087 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.179102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.179191 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.179514 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.179517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.179709 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:49.179823 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.179830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.180869 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.180872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.181653 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.181662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.182202 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.182206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.182319 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.182772 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:49.182786 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:49.183179 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:49.185090 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.185093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.185095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.185352 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.185628 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.185632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.185634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.185734 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.186252 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:49.186255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:49.186342 (+ 87us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.186456 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.186582 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:06:49.202170 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:41047
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:49.307610 10369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:49.307833 10369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:49.307876 10369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:49.311748 10369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:49.311797 10369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:49.311833 10369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 07:06:49.311848 10369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:06:49.311878 10369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:06:49.316905 10369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:41047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:49.317786 10369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:49.318722 10369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:49.327937 10375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:49.327903 10377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:49.327898 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:49.328094 10369 server_base.cc:1034] running on GCE node
I20260620 07:06:49.328593 10369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:49.329144 10369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:49.330294 10369 hybrid_clock.cc:648] HybridClock initialized: now 1781939209330274 us; error 30 us; skew 500 ppm
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:49.332828 10369 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:06:49.333636 10369 webserver.cc:460] Webserver started at http://127.8.24.3:44875/ using document root <none> and password file <none>
I20260620 07:06:49.334149 10369 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:49.334208 10369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:49.334412 10369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:49.336439 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "63b319d3f7ce4e59a103966a74cf20be"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "35e97b433c2bebb15a61f85666acc8d5"
server_key_iv: "fa775fa3d2de5d264b893606f9e35b95"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.336957 10369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "63b319d3f7ce4e59a103966a74cf20be"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "35e97b433c2bebb15a61f85666acc8d5"
server_key_iv: "fa775fa3d2de5d264b893606f9e35b95"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.340859 10369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 07:06:49.343281 10384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:49.344480 10369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:49.344578 10369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "63b319d3f7ce4e59a103966a74cf20be"
format_stamp: "Formatted at 2026-06-20 07:06:49 on dist-test-slave-wjw3"
server_key: "35e97b433c2bebb15a61f85666acc8d5"
server_key_iv: "fa775fa3d2de5d264b893606f9e35b95"
server_key_version: "encryptionkey@0"
I20260620 07:06:49.344678 10369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:49.369977 10369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:49.372823 10369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:49.373029 10369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:49.374519 10369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:49.374571 10369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:49.374611 10369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:49.374639 10369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:49.383966 10369 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:32911
I20260620 07:06:49.384011 10499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:32911 every 8 connection(s)
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:49.384938 10369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:06:49.387951 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.375358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40605 (local address 127.8.24.62:41047)
0620 07:06:49.375497 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.375501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.376581 (+ 1080us) server_negotiation.cc:407] Connection header received
0620 07:06:49.377624 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.377628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.377687 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.377743 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.379386 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.380238 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.381720 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.382049 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.382171 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.384661 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.384665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.384667 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.384695 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.386134 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.386571 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.386574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.386576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.386655 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.386947 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.386950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.386952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.387123 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.387204 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.387678 (+ 474us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.387792 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:06:49.387969 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10369
I20260620 07:06:49.388048 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:06:49.388257 8288 external_mini_cluster.cc:1428] Setting key 1fc351691601c19b704bd27c4c86e2ff
I20260620 07:06:49.388746 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.375989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.3:40605)
0620 07:06:49.376433 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.376475 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.377382 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.378160 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.378169 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.378566 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:49.379215 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.379233 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.380420 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.380424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.381600 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.381609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.382207 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.382211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.382405 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.382983 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:49.383003 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:49.384509 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:49.386233 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.386238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.386252 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.386484 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.386730 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.386733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.386735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.386857 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.387213 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:49.387219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:49.387451 (+ 232us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.387623 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.387911 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":118,"threads_started":1}
I20260620 07:06:49.389863 10500 heartbeater.cc:347] Connected to a master server at 127.8.24.62:41047
I20260620 07:06:49.389999 10500 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:49.390513 10500 heartbeater.cc:511] Master 127.8.24.62:41047 requested a full tablet report, sending...
I20260620 07:06:49.391497 10016 ts_manager.cc:194] Registered new tserver with Master: 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:49.392172 10016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:40605
I20260620 07:06:49.397538 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.391882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:52421 (local address 127.8.24.62:41047)
0620 07:06:49.391984 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.391988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.392136 (+ 148us) server_negotiation.cc:407] Connection header received
0620 07:06:49.392228 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.392232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.392273 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.392337 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.392710 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.393190 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.393954 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.394156 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.394246 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.395113 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.395117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.395118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.395137 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.396166 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.396528 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.396531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.396533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.396598 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.396824 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.396827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.396829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.396991 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.397071 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.397300 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.397398 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 07:06:49.397548 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.391830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.8.24.3:52421)
0620 07:06:49.392067 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.392078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.392161 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.392358 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.392361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.392525 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:49.392611 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.392616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.393296 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.393299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.393856 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.393863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.394253 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.394257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.394356 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.394723 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:49.394732 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:49.394997 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:49.396258 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.396262 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.396264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.396445 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.396672 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.396675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.396677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.396751 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.397075 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:49.397078 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:49.397158 (+ 80us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.397240 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.397377 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260620 07:06:49.401711 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:49.424170 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.417220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.8.24.62:41047)
0620 07:06:49.417333 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.417337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.417351 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:06:49.417391 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.417394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.417434 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.417503 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.418124 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.418627 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.419366 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.419618 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.419691 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.421797 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.421801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.421803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.421823 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.422930 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.423255 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.423257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.423258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.423321 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.423529 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.423531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.423532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.423685 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.423781 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.423933 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.424017 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:06:49.426805 10016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:06:49.429752 10016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:49.442597 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.437145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43601 (local address 127.0.0.1:44490)
0620 07:06:49.437726 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.437776 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.437895 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.438570 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.438575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.438596 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.438687 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.438691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.439843 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.439846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.441050 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.441060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.442065 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.442068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.442168 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.442203 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.442336 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.442459 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260620 07:06:49.442607 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.437283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.8.24.2:43601)
0620 07:06:49.437883 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.437895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.437918 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:06:49.438062 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.438069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.438283 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.438434 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.438793 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.439734 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.441174 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.441962 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.442053 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.442259 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.442372 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.442462 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":102,"threads_started":1}
I20260620 07:06:49.442999 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.437321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.0.0.1:36078)
0620 07:06:49.438210 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.438222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.438316 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.439096 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.439099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.439116 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.439215 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.439221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.440599 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.440603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.441538 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.441547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.442638 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.442641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.442740 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.442762 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.442808 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.442872 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":103,"threads_started":1}
I20260620 07:06:49.443701 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.438254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.8.24.1:34503)
0620 07:06:49.438671 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.438685 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.438706 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:06:49.438774 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.438779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.438964 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.439112 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.439318 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.440488 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.441648 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.442491 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.443249 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.443417 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.443487 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.443573 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":120,"threads_started":1}
I20260620 07:06:49.443914 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.437835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:32911 (local address 127.0.0.1:39028)
0620 07:06:49.438477 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.438488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.438558 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.439137 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.439140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.439156 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.439275 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.439282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.440726 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.440729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.441714 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.441725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.443542 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.443545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.443639 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.443662 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.443730 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.443794 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":11,"thread_start_us":76,"threads_started":1}
I20260620 07:06:49.444005 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.438170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.8.24.3:32911)
0620 07:06:49.438769 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.438779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.438796 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:06:49.438866 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.438870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.439009 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.439417 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.439572 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.440630 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.441841 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.443371 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.443503 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.443640 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.443774 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.443862 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":103,"threads_started":1}
I20260620 07:06:49.446048 10154 tablet_service.cc:1467] Processing CreateTablet for tablet 494f7650644044adb0d9c0da315fc10d (DEFAULT_TABLE table=test-table [id=4a4d1d4758e5405984878e38329af219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.445923 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 494f7650644044adb0d9c0da315fc10d (DEFAULT_TABLE table=test-table [id=4a4d1d4758e5405984878e38329af219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.446916 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494f7650644044adb0d9c0da315fc10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.446939 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494f7650644044adb0d9c0da315fc10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.446640 10421 tablet_service.cc:1467] Processing CreateTablet for tablet 494f7650644044adb0d9c0da315fc10d (DEFAULT_TABLE table=test-table [id=4a4d1d4758e5405984878e38329af219]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.447561 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494f7650644044adb0d9c0da315fc10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.453372 10523 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Bootstrap starting.
I20260620 07:06:49.454980 10525 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Bootstrap starting.
I20260620 07:06:49.454980 10524 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Bootstrap starting.
I20260620 07:06:49.455554 10523 tablet_bootstrap.cc:654] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.456274 10523 log.cc:824] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:49.457007 10525 tablet_bootstrap.cc:654] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.457007 10524 tablet_bootstrap.cc:654] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.457805 10524 log.cc:824] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:49.457805 10525 log.cc:824] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:49.457988 10523 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: No bootstrap required, opened a new log
I20260620 07:06:49.458161 10523 ts_tablet_manager.cc:1397] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260620 07:06:49.459539 10524 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: No bootstrap required, opened a new log
I20260620 07:06:49.459712 10524 ts_tablet_manager.cc:1397] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:06:49.459929 10525 tablet_bootstrap.cc:492] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: No bootstrap required, opened a new log
I20260620 07:06:49.460090 10525 ts_tablet_manager.cc:1397] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:06:49.461364 10523 raft_consensus.cc:348] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.461618 10523 raft_consensus.cc:374] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.461671 10523 raft_consensus.cc:729] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.462060 10523 consensus_queue.cc:260] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.462651 10220 heartbeater.cc:503] Master 127.8.24.62:41047 was elected leader, sending a full tablet report...
I20260620 07:06:49.462908 10523 ts_tablet_manager.cc:1428] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260620 07:06:49.463289 10524 raft_consensus.cc:348] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.463553 10524 raft_consensus.cc:374] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.463608 10524 raft_consensus.cc:729] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.463999 10524 consensus_queue.cc:260] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.464591 10500 heartbeater.cc:503] Master 127.8.24.62:41047 was elected leader, sending a full tablet report...
I20260620 07:06:49.464855 10524 ts_tablet_manager.cc:1428] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:06:49.464705 10525 raft_consensus.cc:348] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.464933 10525 raft_consensus.cc:374] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.465006 10525 raft_consensus.cc:729] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.465369 10525 consensus_queue.cc:260] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.465921 10360 heartbeater.cc:503] Master 127.8.24.62:41047 was elected leader, sending a full tablet report...
I20260620 07:06:49.466460 10525 ts_tablet_manager.cc:1428] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260620 07:06:49.475919 10221 tablet.cc:2369] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:49.502230 10530 raft_consensus.cc:482] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:49.502444 10530 raft_consensus.cc:504] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.503567 10530 leader_election.cc:288] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601), 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503)
I20260620 07:06:49.507112 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.503799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43601 (local address 127.8.24.3:36091)
0620 07:06:49.503942 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.503958 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.504055 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.504377 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.504380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.504399 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.504523 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.504530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.505329 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.505332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.506101 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.506109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.506718 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.506722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.506821 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.506838 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.506900 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.506981 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"spinlock_wait_cycles":896}
I20260620 07:06:49.507112 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.503942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:36091 (local address 127.8.24.2:43601)
0620 07:06:49.504168 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.504171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.504185 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:06:49.504234 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.504237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.504286 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.504353 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.504646 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.505224 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.506218 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.506619 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.506708 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.506770 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.506922 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.506995 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260620 07:06:49.507776 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.503799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.8.24.3:56151)
0620 07:06:49.504268 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.504282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.504462 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.504799 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.504802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.504817 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.504932 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.504938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.505608 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.505611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.506710 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.506719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.507358 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.507362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.507489 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.507510 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.507575 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.507636 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":114,"spinlock_wait_cycles":24704,"thread_start_us":130,"threads_started":1}
I20260620 07:06:49.507771 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.503912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56151 (local address 127.8.24.1:34503)
0620 07:06:49.504077 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.504082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.504352 (+ 270us) server_negotiation.cc:407] Connection header received
0620 07:06:49.504583 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.504586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.504636 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.504699 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.505030 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.505516 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.506821 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.507234 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.507328 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.507384 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.507599 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.507670 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260620 07:06:49.507784 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494f7650644044adb0d9c0da315fc10d" candidate_uuid: "63b319d3f7ce4e59a103966a74cf20be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e45032335974805b36fc5c34a6393c6" is_pre_election: true
I20260620 07:06:49.508069 10314 raft_consensus.cc:2454] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b319d3f7ce4e59a103966a74cf20be in term 0.
I20260620 07:06:49.508385 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494f7650644044adb0d9c0da315fc10d" candidate_uuid: "63b319d3f7ce4e59a103966a74cf20be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" is_pre_election: true
I20260620 07:06:49.508540 10388 leader_election.cc:302] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:49.508678 10174 raft_consensus.cc:2454] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b319d3f7ce4e59a103966a74cf20be in term 0.
I20260620 07:06:49.508817 10530 raft_consensus.cc:2781] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:49.508898 10530 raft_consensus.cc:482] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:49.508949 10530 raft_consensus.cc:3037] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.509967 10530 raft_consensus.cc:504] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.510254 10530 leader_election.cc:288] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [CANDIDATE]: Term 1 election: Requested vote from peers 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601), 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503)
I20260620 07:06:49.510564 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494f7650644044adb0d9c0da315fc10d" candidate_uuid: "63b319d3f7ce4e59a103966a74cf20be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
I20260620 07:06:49.510569 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494f7650644044adb0d9c0da315fc10d" candidate_uuid: "63b319d3f7ce4e59a103966a74cf20be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e45032335974805b36fc5c34a6393c6"
I20260620 07:06:49.510663 10174 raft_consensus.cc:3037] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.510671 10314 raft_consensus.cc:3037] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.511636 10174 raft_consensus.cc:2454] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b319d3f7ce4e59a103966a74cf20be in term 1.
I20260620 07:06:49.511737 10314 raft_consensus.cc:2454] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b319d3f7ce4e59a103966a74cf20be in term 1.
I20260620 07:06:49.511905 10386 leader_election.cc:302] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577a99eafa7b4bdfa90265e862bdc5cd, 63b319d3f7ce4e59a103966a74cf20be; no voters:
I20260620 07:06:49.512099 10530 raft_consensus.cc:2781] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:49.512260 10530 raft_consensus.cc:686] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 1 LEADER]: Becoming Leader. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Running, Role: LEADER
I20260620 07:06:49.512547 10530 consensus_queue.cc:237] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.515671 10016 catalog_manager.cc:5495] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be reported cstate change: term changed from 0 to 1, leader changed from <none> to 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3). New cstate: current_term: 1 leader_uuid: "63b319d3f7ce4e59a103966a74cf20be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } health_report { overall_health: HEALTHY } } }
Jun 20 07:06:49 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:49.548424 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.540981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46260 (local address 127.8.24.62:41047)
0620 07:06:49.541083 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.541088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.541276 (+ 188us) server_negotiation.cc:407] Connection header received
0620 07:06:49.541386 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.541389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.541424 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.541487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:49.542095 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.542617 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.543343 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.543588 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.543666 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.545604 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:49.545607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:49.545610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:49.545633 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:49.546910 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.547328 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.547331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.547332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.547410 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:49.547642 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:49.547644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:49.547645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:49.547801 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:49.547867 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.548152 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.548296 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":33}
I20260620 07:06:49.548435 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.540942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:41047 (local address 127.0.0.1:46260)
0620 07:06:49.541203 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.541214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.541314 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.541516 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.541519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.541904 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:49.541994 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.542000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.542703 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.542706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.543249 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.543255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.543687 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.543690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.543763 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.544212 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:49.544223 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:49.545491 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:49.547004 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.547014 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.547021 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.547261 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.547502 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:49.547504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:49.547506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:49.547580 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:49.547872 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:49.547878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:49.547980 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.548090 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.548233 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":103,"threads_started":1}
I20260620 07:06:49.552045 10016 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46260:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 07:06:49.552703 10016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:49.557212 10294 tablet_service.cc:1467] Processing CreateTablet for tablet b3599a3fd7824aba8b7d2b58624b6dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d96ff6e34344434e999a3dc7578c92ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:49.557062 10154 tablet_service.cc:1467] Processing CreateTablet for tablet b3599a3fd7824aba8b7d2b58624b6dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d96ff6e34344434e999a3dc7578c92ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:49.557330 10421 tablet_service.cc:1467] Processing CreateTablet for tablet b3599a3fd7824aba8b7d2b58624b6dea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d96ff6e34344434e999a3dc7578c92ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:06:49.557778 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3599a3fd7824aba8b7d2b58624b6dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.557780 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3599a3fd7824aba8b7d2b58624b6dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.557829 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3599a3fd7824aba8b7d2b58624b6dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.560098 10525 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: Bootstrap starting.
I20260620 07:06:49.560454 10524 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: Bootstrap starting.
I20260620 07:06:49.560678 10523 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: Bootstrap starting.
I20260620 07:06:49.561132 10525 tablet_bootstrap.cc:654] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.561342 10524 tablet_bootstrap.cc:654] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.561497 10523 tablet_bootstrap.cc:654] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.562532 10523 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: No bootstrap required, opened a new log
I20260620 07:06:49.562569 10524 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: No bootstrap required, opened a new log
I20260620 07:06:49.562625 10523 ts_tablet_manager.cc:1397] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260620 07:06:49.562640 10524 ts_tablet_manager.cc:1397] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:49.562927 10525 tablet_bootstrap.cc:492] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: No bootstrap required, opened a new log
I20260620 07:06:49.563007 10525 ts_tablet_manager.cc:1397] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 07:06:49.563001 10524 raft_consensus.cc:348] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563113 10524 raft_consensus.cc:374] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.563061 10523 raft_consensus.cc:348] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563148 10524 raft_consensus.cc:729] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.563158 10523 raft_consensus.cc:374] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.563186 10523 raft_consensus.cc:729] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.563269 10524 consensus_queue.cc:260] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563303 10523 consensus_queue.cc:260] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563423 10525 raft_consensus.cc:348] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563535 10524 ts_tablet_manager.cc:1428] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:49.563545 10525 raft_consensus.cc:374] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.563576 10525 raft_consensus.cc:729] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.563584 10523 ts_tablet_manager.cc:1428] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:49.563694 10525 consensus_queue.cc:260] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.563726 10530 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.563836 10530 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.563781 10529 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.563905 10529 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.563922 10531 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.564019 10531 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.564523 10525 ts_tablet_manager.cc:1428] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 07:06:49.593547 10529 raft_consensus.cc:482] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:49.593657 10529 raft_consensus.cc:504] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.594692 10529 leader_election.cc:288] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:49.598798 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.595018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:51297 (local address 127.8.24.3:32911)
0620 07:06:49.595214 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.595218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.595299 (+ 81us) server_negotiation.cc:407] Connection header received
0620 07:06:49.595397 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.595400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.595468 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.595541 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.595902 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.596453 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.597762 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.598232 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.598325 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.598386 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.598626 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.598695 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260620 07:06:49.598824 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.594911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:32911 (local address 127.8.24.1:51297)
0620 07:06:49.595227 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.595238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.595312 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.595557 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.595560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.595598 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.595807 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.595816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.596545 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.596548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.597657 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.597667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.598352 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.598356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.598464 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.598490 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.598593 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.598666 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":102,"spinlock_wait_cycles":34048,"thread_start_us":71,"threads_started":1}
I20260620 07:06:49.599165 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.595011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34321 (local address 127.8.24.2:43601)
0620 07:06:49.595159 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.595164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.595260 (+ 96us) server_negotiation.cc:407] Connection header received
0620 07:06:49.595395 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.595398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.595468 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.595538 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:49.596293 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.596826 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.598029 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.598544 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.598634 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.598688 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.599002 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.599066 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 07:06:49.599182 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.594911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43601 (local address 127.8.24.1:34321)
0620 07:06:49.595186 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.595201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.595311 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.595557 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.595560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.595600 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:49.596189 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.596197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.596934 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.596937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.597938 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.597945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.598748 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.598752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.598851 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.598874 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.598971 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.599037 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":38,"spinlock_wait_cycles":9344,"thread_start_us":90,"threads_started":1}
I20260620 07:06:49.599539 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3599a3fd7824aba8b7d2b58624b6dea" candidate_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be" is_pre_election: true
I20260620 07:06:49.599567 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3599a3fd7824aba8b7d2b58624b6dea" candidate_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e45032335974805b36fc5c34a6393c6" is_pre_election: true
I20260620 07:06:49.599704 10314 raft_consensus.cc:2454] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 577a99eafa7b4bdfa90265e862bdc5cd in term 0.
I20260620 07:06:49.599763 10445 raft_consensus.cc:2454] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 577a99eafa7b4bdfa90265e862bdc5cd in term 0.
I20260620 07:06:49.600018 10108 leader_election.cc:302] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577a99eafa7b4bdfa90265e862bdc5cd, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:49.600265 10529 raft_consensus.cc:2781] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:49.600344 10529 raft_consensus.cc:482] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:49.600389 10529 raft_consensus.cc:3037] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.601318 10529 raft_consensus.cc:504] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.601608 10529 leader_election.cc:288] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [CANDIDATE]: Term 1 election: Requested vote from peers 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:49.601922 10314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3599a3fd7824aba8b7d2b58624b6dea" candidate_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e45032335974805b36fc5c34a6393c6"
I20260620 07:06:49.601945 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3599a3fd7824aba8b7d2b58624b6dea" candidate_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be"
I20260620 07:06:49.602035 10314 raft_consensus.cc:3037] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.602035 10445 raft_consensus.cc:3037] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:49.603049 10314 raft_consensus.cc:2454] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 577a99eafa7b4bdfa90265e862bdc5cd in term 1.
I20260620 07:06:49.603049 10445 raft_consensus.cc:2454] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 577a99eafa7b4bdfa90265e862bdc5cd in term 1.
I20260620 07:06:49.603299 10108 leader_election.cc:302] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 577a99eafa7b4bdfa90265e862bdc5cd, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:49.603508 10529 raft_consensus.cc:2781] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:49.603713 10529 raft_consensus.cc:686] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 LEADER]: Becoming Leader. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Running, Role: LEADER
I20260620 07:06:49.604048 10529 consensus_queue.cc:237] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.606134 10546 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: TxnStatusTablet state changed. Reason: New leader 577a99eafa7b4bdfa90265e862bdc5cd. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.606240 10546 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:49.606503 10548 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 07:06:49.606699 10548 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 07:06:49.606966 10014 catalog_manager.cc:5495] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1). New cstate: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:49.630327 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.626940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.0.0.1:36088)
0620 07:06:49.627078 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.627094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.627193 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.627494 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.627497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.627505 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:49.627600 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.627605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.628274 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.628278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.628883 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.628898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.629315 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.629319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.629402 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.629441 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.630045 (+ 604us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.630060 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.630124 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.630205 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 07:06:49.630327 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.627018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.8.24.1:34503)
0620 07:06:49.627124 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.627129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.627141 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:06:49.627325 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.627327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.627379 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.627486 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:49.627690 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.628174 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.629005 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.629219 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.629303 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.629565 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.629954 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.630047 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.630140 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.630192 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:06:49.635542 10314 raft_consensus.cc:1261] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Refusing update from remote peer 577a99eafa7b4bdfa90265e862bdc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:49.635565 10445 raft_consensus.cc:1261] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Refusing update from remote peer 577a99eafa7b4bdfa90265e862bdc5cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260620 07:06:49.636026 10501 tablet.cc:2369] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:49.636169 10546 consensus_queue.cc:1035] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:49.636333 10529 consensus_queue.cc:1035] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:49.639101 10530 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: TxnStatusTablet state changed. Reason: New leader 577a99eafa7b4bdfa90265e862bdc5cd. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.639218 10530 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.639806 10531 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: TxnStatusTablet state changed. Reason: New leader 577a99eafa7b4bdfa90265e862bdc5cd. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.639926 10531 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.643791 10529 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.643905 10529 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:49.644022 10529 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.644091 10529 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 577a99eafa7b4bdfa90265e862bdc5cd: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:06:49.644151 10530 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.644236 10530 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 63b319d3f7ce4e59a103966a74cf20be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.644484 10531 tablet_replica.cc:439] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } } }
I20260620 07:06:49.644586 10531 tablet_replica.cc:442] T b3599a3fd7824aba8b7d2b58624b6dea P 7e45032335974805b36fc5c34a6393c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:06:49.645138 10552 mvcc.cc:205] Tried to move back new op lower bound from 7298823002660700160 to 7298823002540695552. Current Snapshot: MvccSnapshot[applied={T|T < 7298823002660700160}]
I20260620 07:06:49.644802 10551 mvcc.cc:205] Tried to move back new op lower bound from 7298823002660700160 to 7298823002540695552. Current Snapshot: MvccSnapshot[applied={T|T < 7298823002660700160}]
I20260620 07:06:49.646752 10553 mvcc.cc:205] Tried to move back new op lower bound from 7298823002660700160 to 7298823002540695552. Current Snapshot: MvccSnapshot[applied={T|T < 7298823002660700160}]
I20260620 07:06:49.654913 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.651229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.8.24.3:32911)
0620 07:06:49.651340 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.651344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.651358 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:06:49.651494 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.651497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.651553 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.651649 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:49.651837 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.652647 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.653485 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.653845 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.653946 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.654135 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.654608 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.654710 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.654739 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.654788 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:06:49.664238 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.661450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:52709 (local address 127.8.24.1:34503)
0620 07:06:49.661556 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.661561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.661574 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:06:49.661638 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.661641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.661684 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.661741 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:49.661962 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.662447 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.663255 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.663520 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.663605 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.663862 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.663929 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.663985 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.664066 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.664112 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:06:49.664237 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.661360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.8.24.3:52709)
0620 07:06:49.661477 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.661490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.661564 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.661759 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.661761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.661767 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:49.661861 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.661866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.662543 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.662546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.663146 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.663153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.663609 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.663612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.663694 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.663730 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.663995 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.664001 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.664056 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.664125 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 07:06:49.670698 10314 raft_consensus.cc:1261] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Refusing update from remote peer 63b319d3f7ce4e59a103966a74cf20be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:49.670815 10174 raft_consensus.cc:1261] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Refusing update from remote peer 63b319d3f7ce4e59a103966a74cf20be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:49.671146 10533 consensus_queue.cc:1035] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:49.671294 10530 consensus_queue.cc:1035] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [LEADER]: Connected to new peer: Peer: permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 07:06:49.672698 10361 tablet.cc:2369] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:06:49.694131 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.690458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:32911 (local address 127.8.24.1:55827)
0620 07:06:49.690764 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:49.690780 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:49.690905 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:49.691122 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:49.691124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:49.691132 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:49.691231 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.691237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.691935 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.691938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.693064 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:49.693075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.693502 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.693505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:49.693590 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.693631 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.693899 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.693905 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:06:49.693945 (+ 40us) client_negotiation.cc:240] Negotiation successful
0620 07:06:49.694006 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
I20260620 07:06:49.694133 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:49.690768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55827 (local address 127.8.24.3:32911)
0620 07:06:49.690868 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:49.690872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:49.690886 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:06:49.690995 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:49.690998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:49.691043 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:49.691100 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:49.691311 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.691839 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.693152 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:49.693406 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:49.693494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:49.693745 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:49.693823 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:49.693884 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:49.693961 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 07:06:49.694010 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:06:49.715517 10014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:49.715714 10014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:49.718246 10014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Sending DeleteTablet for 3 replicas of tablet 494f7650644044adb0d9c0da315fc10d
I20260620 07:06:49.718889 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 494f7650644044adb0d9c0da315fc10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260620 07:06:49.719127 10421 tablet_service.cc:1514] Processing DeleteTablet for tablet 494f7650644044adb0d9c0da315fc10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39028
I20260620 07:06:49.719708 10154 tablet_service.cc:1514] Processing DeleteTablet for tablet 494f7650644044adb0d9c0da315fc10d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:49 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36078
I20260620 07:06:49.720299 10014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 07:06:49.720625 10570 tablet_replica.cc:330] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: stopping tablet replica
W20260620 07:06:49.720718 10014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:49.721028 10570 raft_consensus.cc:2229] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:49.721288 10570 raft_consensus.cc:2258] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:49.722596 10570 ts_tablet_manager.cc:1905] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:49.724174 10571 tablet_replica.cc:330] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: stopping tablet replica
I20260620 07:06:49.724468 10571 raft_consensus.cc:2229] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:49.725693 10294 tablet_service.cc:1467] Processing CreateTablet for tablet b1d21e0fba514fbbb07529362de771b1 (DEFAULT_TABLE table=test-table [id=3a91e80a10614d168958145c509ff16a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.725950 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1d21e0fba514fbbb07529362de771b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.726173 10571 raft_consensus.cc:2258] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:49.727548 10571 ts_tablet_manager.cc:1905] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:49.728384 10525 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Bootstrap starting.
I20260620 07:06:49.729274 10525 tablet_bootstrap.cc:654] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.730109 10570 ts_tablet_manager.cc:1918] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:49.730511 10421 tablet_service.cc:1467] Processing CreateTablet for tablet b1d21e0fba514fbbb07529362de771b1 (DEFAULT_TABLE table=test-table [id=3a91e80a10614d168958145c509ff16a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.730702 10570 log.cc:1196] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/494f7650644044adb0d9c0da315fc10d
I20260620 07:06:49.730764 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1d21e0fba514fbbb07529362de771b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.732292 10571 ts_tablet_manager.cc:1918] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:49.732664 10570 ts_tablet_manager.cc:1939] T 494f7650644044adb0d9c0da315fc10d P 7e45032335974805b36fc5c34a6393c6: Deleting consensus metadata
I20260620 07:06:49.733923 10004 catalog_manager.cc:4841] TS 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601): tablet 494f7650644044adb0d9c0da315fc10d (table test-table [id=4a4d1d4758e5405984878e38329af219]) successfully deleted
I20260620 07:06:49.732981 10524 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Bootstrap starting.
I20260620 07:06:49.735013 10524 tablet_bootstrap.cc:654] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.734966 10154 tablet_service.cc:1467] Processing CreateTablet for tablet b1d21e0fba514fbbb07529362de771b1 (DEFAULT_TABLE table=test-table [id=3a91e80a10614d168958145c509ff16a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:49.735260 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1d21e0fba514fbbb07529362de771b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:49.735558 10571 log.cc:1196] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/494f7650644044adb0d9c0da315fc10d
I20260620 07:06:49.735973 10571 ts_tablet_manager.cc:1939] T 494f7650644044adb0d9c0da315fc10d P 63b319d3f7ce4e59a103966a74cf20be: Deleting consensus metadata
I20260620 07:06:49.736542 10525 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: No bootstrap required, opened a new log
I20260620 07:06:49.736630 10525 ts_tablet_manager.cc:1397] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260620 07:06:49.737053 10525 raft_consensus.cc:348] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.737104 10002 catalog_manager.cc:4841] TS 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911): tablet 494f7650644044adb0d9c0da315fc10d (table test-table [id=4a4d1d4758e5405984878e38329af219]) successfully deleted
I20260620 07:06:49.737164 10525 raft_consensus.cc:374] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.737197 10525 raft_consensus.cc:729] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.737317 10525 consensus_queue.cc:260] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.737567 10525 ts_tablet_manager.cc:1428] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 07:06:49.737710 10523 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Bootstrap starting.
I20260620 07:06:49.738940 10523 tablet_bootstrap.cc:654] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:49.740124 10523 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: No bootstrap required, opened a new log
I20260620 07:06:49.740207 10523 ts_tablet_manager.cc:1397] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 07:06:49.740653 10523 raft_consensus.cc:348] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.740780 10523 raft_consensus.cc:374] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.740815 10523 raft_consensus.cc:729] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.740934 10523 consensus_queue.cc:260] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.741787 10523 ts_tablet_manager.cc:1428] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:49.744030 10524 tablet_bootstrap.cc:492] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: No bootstrap required, opened a new log
I20260620 07:06:49.744103 10524 ts_tablet_manager.cc:1397] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260620 07:06:49.744486 10524 raft_consensus.cc:348] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.744580 10524 raft_consensus.cc:374] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:49.744604 10524 raft_consensus.cc:729] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Initialized, Role: FOLLOWER
I20260620 07:06:49.744714 10524 consensus_queue.cc:260] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:49.744938 10524 ts_tablet_manager.cc:1428] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:49.751631 10572 tablet_replica.cc:330] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: stopping tablet replica
I20260620 07:06:49.751989 10572 raft_consensus.cc:2229] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:49.752172 10572 raft_consensus.cc:2258] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:49.753510 10572 ts_tablet_manager.cc:1905] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:49.756672 10572 ts_tablet_manager.cc:1918] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:49.756775 10572 log.cc:1196] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/494f7650644044adb0d9c0da315fc10d
I20260620 07:06:49.757377 10572 ts_tablet_manager.cc:1939] T 494f7650644044adb0d9c0da315fc10d P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting consensus metadata
I20260620 07:06:49.758550 10002 catalog_manager.cc:4841] TS 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503): tablet 494f7650644044adb0d9c0da315fc10d (table test-table [id=4a4d1d4758e5405984878e38329af219]) successfully deleted
I20260620 07:06:50.031421 10531 raft_consensus.cc:482] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:50.031574 10531 raft_consensus.cc:504] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:50.032572 10531 leader_election.cc:288] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:50.036469 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.032903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:47205 (local address 127.8.24.1:34503)
0620 07:06:50.033035 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:50.033039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:50.033194 (+ 155us) server_negotiation.cc:407] Connection header received
0620 07:06:50.033317 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:50.033320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:50.033368 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:50.033452 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:50.033741 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.034331 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.035549 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.035921 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.035999 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.036062 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:50.036272 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 07:06:50.036339 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:06:50.036468 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.033275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48619 (local address 127.8.24.3:32911)
0620 07:06:50.033391 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:50.033404 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:50.033512 (+ 108us) server_negotiation.cc:407] Connection header received
0620 07:06:50.033621 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:50.033624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:50.033669 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:50.033738 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:06:50.033961 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.034456 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.035561 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.035946 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.036041 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.036099 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:50.036289 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 07:06:50.036353 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 07:06:50.036489 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.033174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:32911 (local address 127.8.24.2:48619)
0620 07:06:50.033442 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:50.033454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:50.033544 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:50.033749 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:50.033752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:50.033768 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:50.033869 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.033874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.034555 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.034557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.035405 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.035413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.036076 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.036080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.036155 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.036195 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 07:06:50.036261 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:06:50.036349 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":57,"thread_start_us":52,"threads_started":1}
I20260620 07:06:50.036491 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.032816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.8.24.2:47205)
0620 07:06:50.033100 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:50.033114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:50.033219 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:50.033469 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:50.033471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:50.033491 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:06:50.033632 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.033639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.034429 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.034432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.035408 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.035417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.036028 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.036032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.036143 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.036181 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 07:06:50.036237 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 07:06:50.036332 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":30,"spinlock_wait_cycles":9984,"thread_start_us":81,"threads_started":1}
I20260620 07:06:50.036854 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d21e0fba514fbbb07529362de771b1" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be" is_pre_election: true
I20260620 07:06:50.037007 10445 raft_consensus.cc:2454] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:50.036983 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d21e0fba514fbbb07529362de771b1" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" is_pre_election: true
I20260620 07:06:50.037092 10174 raft_consensus.cc:2454] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:50.037380 10246 leader_election.cc:302] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:50.037607 10531 raft_consensus.cc:2781] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:50.037658 10531 raft_consensus.cc:482] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:50.037683 10531 raft_consensus.cc:3037] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:50.038594 10531 raft_consensus.cc:504] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:50.038883 10531 leader_election.cc:288] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Requested vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:50.039211 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d21e0fba514fbbb07529362de771b1" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be"
I20260620 07:06:50.039302 10445 raft_consensus.cc:3037] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:50.039245 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d21e0fba514fbbb07529362de771b1" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
I20260620 07:06:50.039393 10174 raft_consensus.cc:3037] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:50.040151 10445 raft_consensus.cc:2454] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:50.040288 10174 raft_consensus.cc:2454] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:50.040395 10246 leader_election.cc:302] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:50.040544 10531 raft_consensus.cc:2781] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:50.040735 10531 raft_consensus.cc:686] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Running, Role: LEADER
I20260620 07:06:50.040974 10531 consensus_queue.cc:237] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:50.043201 10016 catalog_manager.cc:5495] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e45032335974805b36fc5c34a6393c6 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "7e45032335974805b36fc5c34a6393c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:50.112305 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.109387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.8.24.2:43601)
0620 07:06:50.109571 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:50.109574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:50.109585 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:06:50.109646 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:50.109648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:50.109696 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:50.109788 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:50.109935 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.110445 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.111133 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.111336 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.111406 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.111617 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:50.112041 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:50.112125 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:50.112151 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 07:06:50.112193 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":58,"threads_started":1}
I20260620 07:06:50.121274 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.118578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:39023 (local address 127.8.24.1:34503)
0620 07:06:50.118679 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:50.118683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:50.118856 (+ 173us) server_negotiation.cc:407] Connection header received
0620 07:06:50.118944 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:50.118946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:50.118984 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:50.119040 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:50.119197 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.119713 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.120479 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.120675 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.120737 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.120927 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:50.120985 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:50.121042 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:50.121134 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 07:06:50.121174 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:06:50.121321 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.118506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.8.24.2:39023)
0620 07:06:50.118797 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:50.118807 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:50.118885 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:50.119049 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:50.119051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:50.119056 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:50.119142 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.119145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.119782 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.119783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.120401 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.120409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.120742 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.120744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.120805 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.120840 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:50.121045 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:50.121049 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:06:50.121125 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 07:06:50.121185 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":83,"threads_started":1}
I20260620 07:06:50.126487 10445 raft_consensus.cc:1261] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:50.126546 10174 raft_consensus.cc:1261] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:50.126886 10531 consensus_queue.cc:1035] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:50.127045 10578 consensus_queue.cc:1035] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:50.142127 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.139016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43601 (local address 127.8.24.1:47147)
0620 07:06:50.139123 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:50.139137 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:50.139210 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:50.139459 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:50.139462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:50.139471 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:06:50.139567 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.139573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.140403 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.140406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.140985 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:50.140994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.141440 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.141444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:50.141526 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.141577 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:06:50.141835 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:06:50.141842 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 07:06:50.141910 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:06:50.141992 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 07:06:50.142153 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:50.139081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:47147 (local address 127.8.24.2:43601)
0620 07:06:50.139185 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:50.139189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:50.139202 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:06:50.139306 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:50.139309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:50.139349 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:50.139410 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:50.139666 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.140307 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.141113 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:50.141330 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:50.141442 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:50.141675 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:50.141751 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:50.141817 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:50.141954 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 07:06:50.142010 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 07:06:50.142829 10552 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 07:06:51.694562 10016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:51.694706 10016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:51.696739 10016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Sending DeleteTablet for 3 replicas of tablet b1d21e0fba514fbbb07529362de771b1
I20260620 07:06:51.697196 10154 tablet_service.cc:1514] Processing DeleteTablet for tablet b1d21e0fba514fbbb07529362de771b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36078
I20260620 07:06:51.697229 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet b1d21e0fba514fbbb07529362de771b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260620 07:06:51.697255 10421 tablet_service.cc:1514] Processing DeleteTablet for tablet b1d21e0fba514fbbb07529362de771b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39028
I20260620 07:06:51.697520 10613 tablet_replica.cc:330] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: stopping tablet replica
I20260620 07:06:51.697614 10613 raft_consensus.cc:2229] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:51.697688 10613 raft_consensus.cc:2258] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.698186 10613 ts_tablet_manager.cc:1905] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:51.700491 10613 ts_tablet_manager.cc:1918] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 07:06:51.700523 10611 tablet_replica.cc:330] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: stopping tablet replica
I20260620 07:06:51.700630 10613 log.cc:1196] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/b1d21e0fba514fbbb07529362de771b1
I20260620 07:06:51.700628 10611 raft_consensus.cc:2229] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:51.700713 10611 raft_consensus.cc:2258] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.700906 10613 ts_tablet_manager.cc:1939] T b1d21e0fba514fbbb07529362de771b1 P 63b319d3f7ce4e59a103966a74cf20be: Deleting consensus metadata
I20260620 07:06:51.701228 10611 ts_tablet_manager.cc:1905] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:51.701467 10612 tablet_replica.cc:330] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: stopping tablet replica
I20260620 07:06:51.701445 10002 catalog_manager.cc:4841] TS 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911): tablet b1d21e0fba514fbbb07529362de771b1 (table test-table [id=3a91e80a10614d168958145c509ff16a]) successfully deleted
I20260620 07:06:51.701627 10612 raft_consensus.cc:2229] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:51.701943 10612 raft_consensus.cc:2258] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.702401 10612 ts_tablet_manager.cc:1905] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:51.703603 10611 ts_tablet_manager.cc:1918] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 07:06:51.703722 10611 log.cc:1196] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/b1d21e0fba514fbbb07529362de771b1
I20260620 07:06:51.703982 10611 ts_tablet_manager.cc:1939] T b1d21e0fba514fbbb07529362de771b1 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting consensus metadata
I20260620 07:06:51.704548 10002 catalog_manager.cc:4841] TS 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503): tablet b1d21e0fba514fbbb07529362de771b1 (table test-table [id=3a91e80a10614d168958145c509ff16a]) successfully deleted
I20260620 07:06:51.704767 10612 ts_tablet_manager.cc:1918] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 07:06:51.704828 10612 log.cc:1196] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/b1d21e0fba514fbbb07529362de771b1
I20260620 07:06:51.705080 10612 ts_tablet_manager.cc:1939] T b1d21e0fba514fbbb07529362de771b1 P 7e45032335974805b36fc5c34a6393c6: Deleting consensus metadata
I20260620 07:06:51.705595 10004 catalog_manager.cc:4841] TS 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601): tablet b1d21e0fba514fbbb07529362de771b1 (table test-table [id=3a91e80a10614d168958145c509ff16a]) successfully deleted
Jun 20 07:06:51 dist-test-slave-wjw3 krb5kdc[9972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939211, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 07:06:51 dist-test-slave-wjw3 krb5kdc[9972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939211, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:51.725361 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:51.713870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.8.24.62:41047)
0620 07:06:51.714120 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:51.714127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:51.714140 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:06:51.714232 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:51.714234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:51.714284 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:51.714352 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:51.714923 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:51.715478 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:51.716225 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:51.716436 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:51.716509 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:51.722457 (+ 5948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:51.722460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:51.722462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:51.722485 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:51.723913 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:51.724273 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:51.724276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:51.724278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:51.724354 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:51.724583 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:51.724587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:51.724588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:51.724762 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:51.724874 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:51.725094 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 07:06:51.725208 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":92,"threads_started":1}
I20260620 07:06:51.727540 10016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:06:51.727918 10016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:51.732003 10154 tablet_service.cc:1467] Processing CreateTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 (DEFAULT_TABLE table=test-table [id=48e7a72e667941609fd48925200000da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.732182 10421 tablet_service.cc:1467] Processing CreateTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 (DEFAULT_TABLE table=test-table [id=48e7a72e667941609fd48925200000da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.732199 10294 tablet_service.cc:1467] Processing CreateTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 (DEFAULT_TABLE table=test-table [id=48e7a72e667941609fd48925200000da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.732261 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f812414ba8b04ebfa49ebc3c0645a627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.732424 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f812414ba8b04ebfa49ebc3c0645a627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.732424 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f812414ba8b04ebfa49ebc3c0645a627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.734704 10623 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Bootstrap starting.
I20260620 07:06:51.734802 10624 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Bootstrap starting.
I20260620 07:06:51.734915 10622 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Bootstrap starting.
I20260620 07:06:51.735616 10624 tablet_bootstrap.cc:654] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.735616 10623 tablet_bootstrap.cc:654] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.735662 10622 tablet_bootstrap.cc:654] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.736636 10624 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: No bootstrap required, opened a new log
I20260620 07:06:51.736681 10622 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: No bootstrap required, opened a new log
I20260620 07:06:51.736761 10624 ts_tablet_manager.cc:1397] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 07:06:51.736769 10622 ts_tablet_manager.cc:1397] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:51.737267 10624 raft_consensus.cc:348] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.737303 10622 raft_consensus.cc:348] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.737358 10624 raft_consensus.cc:374] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.737385 10622 raft_consensus.cc:374] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.737388 10624 raft_consensus.cc:729] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.737422 10622 raft_consensus.cc:729] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.737504 10624 consensus_queue.cc:260] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.737534 10622 consensus_queue.cc:260] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.737751 10622 ts_tablet_manager.cc:1428] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:51.737848 10624 ts_tablet_manager.cc:1428] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:51.738598 10623 tablet_bootstrap.cc:492] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: No bootstrap required, opened a new log
I20260620 07:06:51.738689 10623 ts_tablet_manager.cc:1397] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 07:06:51.739126 10623 raft_consensus.cc:348] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.739243 10623 raft_consensus.cc:374] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.739282 10623 raft_consensus.cc:729] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.739403 10623 consensus_queue.cc:260] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.739701 10623 ts_tablet_manager.cc:1428] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:51.859102 10605 raft_consensus.cc:482] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:51.859239 10605 raft_consensus.cc:504] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.859709 10605 leader_election.cc:288] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:51.860225 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f812414ba8b04ebfa49ebc3c0645a627" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" is_pre_election: true
I20260620 07:06:51.860329 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f812414ba8b04ebfa49ebc3c0645a627" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be" is_pre_election: true
I20260620 07:06:51.860471 10445 raft_consensus.cc:2454] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:51.860466 10174 raft_consensus.cc:2454] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:51.860786 10246 leader_election.cc:302] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:51.860952 10605 raft_consensus.cc:2781] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:51.861014 10605 raft_consensus.cc:482] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:51.861052 10605 raft_consensus.cc:3037] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:51.862015 10605 raft_consensus.cc:504] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.862300 10605 leader_election.cc:288] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Requested vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:51.862638 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f812414ba8b04ebfa49ebc3c0645a627" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
I20260620 07:06:51.862646 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f812414ba8b04ebfa49ebc3c0645a627" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be"
I20260620 07:06:51.862740 10445 raft_consensus.cc:3037] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:51.862743 10174 raft_consensus.cc:3037] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:51.863593 10445 raft_consensus.cc:2454] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:51.863677 10174 raft_consensus.cc:2454] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:51.863837 10246 leader_election.cc:302] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:51.863996 10605 raft_consensus.cc:2781] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:51.864100 10605 raft_consensus.cc:686] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Running, Role: LEADER
I20260620 07:06:51.864234 10605 consensus_queue.cc:237] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.865787 10017 catalog_manager.cc:5495] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e45032335974805b36fc5c34a6393c6 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "7e45032335974805b36fc5c34a6393c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:51.893975 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:51.891300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.8.24.2:43601)
0620 07:06:51.891570 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:51.891576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:51.891587 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:06:51.891622 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:51.891624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:51.891661 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:51.891724 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:06:51.891931 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:51.892435 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:51.893185 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:51.893400 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:51.893461 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:51.893636 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:06:51.893697 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:06:51.893741 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:51.893798 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 07:06:51.893837 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":144,"threads_started":1}
I20260620 07:06:51.899205 10174 raft_consensus.cc:1261] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:51.899363 10445 raft_consensus.cc:1261] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:51.899737 10605 consensus_queue.cc:1035] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:51.899930 10586 consensus_queue.cc:1035] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:51.929086 10017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:51.929229 10017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:51.930979 10017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Sending DeleteTablet for 3 replicas of tablet f812414ba8b04ebfa49ebc3c0645a627
I20260620 07:06:51.931427 10154 tablet_service.cc:1514] Processing DeleteTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36078
I20260620 07:06:51.931593 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260620 07:06:51.931632 10421 tablet_service.cc:1514] Processing DeleteTablet for tablet f812414ba8b04ebfa49ebc3c0645a627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:51 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39028
I20260620 07:06:51.931694 10611 tablet_replica.cc:330] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: stopping tablet replica
I20260620 07:06:51.931746 10612 tablet_replica.cc:330] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: stopping tablet replica
I20260620 07:06:51.931784 10613 tablet_replica.cc:330] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: stopping tablet replica
I20260620 07:06:51.931808 10611 raft_consensus.cc:2229] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:51.931824 10612 raft_consensus.cc:2229] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:51.931880 10613 raft_consensus.cc:2229] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:51.931905 10611 raft_consensus.cc:2258] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.931962 10613 raft_consensus.cc:2258] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.931974 10612 raft_consensus.cc:2258] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:51.932392 10017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 07:06:51.932674 10612 ts_tablet_manager.cc:1905] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 07:06:51.932770 10017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:06:51.932832 10611 ts_tablet_manager.cc:1905] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:51.933727 10613 ts_tablet_manager.cc:1905] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:51.935134 10611 ts_tablet_manager.cc:1918] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:51.935173 10612 ts_tablet_manager.cc:1918] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:51.935217 10611 log.cc:1196] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/f812414ba8b04ebfa49ebc3c0645a627
I20260620 07:06:51.935249 10612 log.cc:1196] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/f812414ba8b04ebfa49ebc3c0645a627
I20260620 07:06:51.935474 10611 ts_tablet_manager.cc:1939] T f812414ba8b04ebfa49ebc3c0645a627 P 577a99eafa7b4bdfa90265e862bdc5cd: Deleting consensus metadata
I20260620 07:06:51.935745 10612 ts_tablet_manager.cc:1939] T f812414ba8b04ebfa49ebc3c0645a627 P 7e45032335974805b36fc5c34a6393c6: Deleting consensus metadata
I20260620 07:06:51.936097 10613 ts_tablet_manager.cc:1918] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:06:51.936146 10613 log.cc:1196] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/f812414ba8b04ebfa49ebc3c0645a627
I20260620 07:06:51.936348 10613 ts_tablet_manager.cc:1939] T f812414ba8b04ebfa49ebc3c0645a627 P 63b319d3f7ce4e59a103966a74cf20be: Deleting consensus metadata
I20260620 07:06:51.936384 10004 catalog_manager.cc:4841] TS 7e45032335974805b36fc5c34a6393c6 (127.8.24.2:43601): tablet f812414ba8b04ebfa49ebc3c0645a627 (table test-table [id=48e7a72e667941609fd48925200000da]) successfully deleted
I20260620 07:06:51.936667 10002 catalog_manager.cc:4841] TS 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503): tablet f812414ba8b04ebfa49ebc3c0645a627 (table test-table [id=48e7a72e667941609fd48925200000da]) successfully deleted
I20260620 07:06:51.937362 10294 tablet_service.cc:1467] Processing CreateTablet for tablet 6d08c0332a2345aba386be92b1c5d50d (DEFAULT_TABLE table=test-table [id=08d20168346748bea9bd1d31ff115785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.937467 10154 tablet_service.cc:1467] Processing CreateTablet for tablet 6d08c0332a2345aba386be92b1c5d50d (DEFAULT_TABLE table=test-table [id=08d20168346748bea9bd1d31ff115785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.937474 10421 tablet_service.cc:1467] Processing CreateTablet for tablet 6d08c0332a2345aba386be92b1c5d50d (DEFAULT_TABLE table=test-table [id=08d20168346748bea9bd1d31ff115785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:06:51.937597 10294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d08c0332a2345aba386be92b1c5d50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.937652 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d08c0332a2345aba386be92b1c5d50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.937703 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d08c0332a2345aba386be92b1c5d50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:51.939301 10624 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Bootstrap starting.
I20260620 07:06:51.939574 10623 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: Bootstrap starting.
I20260620 07:06:51.940099 10624 tablet_bootstrap.cc:654] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.940248 10623 tablet_bootstrap.cc:654] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.941023 10622 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: Bootstrap starting.
I20260620 07:06:51.941027 10624 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: No bootstrap required, opened a new log
I20260620 07:06:51.941103 10624 ts_tablet_manager.cc:1397] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:51.941298 10002 catalog_manager.cc:4841] TS 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911): tablet f812414ba8b04ebfa49ebc3c0645a627 (table test-table [id=48e7a72e667941609fd48925200000da]) successfully deleted
I20260620 07:06:51.941479 10624 raft_consensus.cc:348] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.941604 10624 raft_consensus.cc:374] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.941640 10624 raft_consensus.cc:729] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b319d3f7ce4e59a103966a74cf20be, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.941779 10622 tablet_bootstrap.cc:654] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:51.941758 10624 consensus_queue.cc:260] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.941934 10623 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: No bootstrap required, opened a new log
I20260620 07:06:51.942003 10623 ts_tablet_manager.cc:1397] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:51.942021 10624 ts_tablet_manager.cc:1428] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:06:51.942368 10623 raft_consensus.cc:348] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.942480 10623 raft_consensus.cc:374] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.942513 10623 raft_consensus.cc:729] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.942628 10623 consensus_queue.cc:260] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.942874 10623 ts_tablet_manager.cc:1428] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:51.943671 10622 tablet_bootstrap.cc:492] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: No bootstrap required, opened a new log
I20260620 07:06:51.943755 10622 ts_tablet_manager.cc:1397] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 07:06:51.944131 10622 raft_consensus.cc:348] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.944226 10622 raft_consensus.cc:374] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:51.944257 10622 raft_consensus.cc:729] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 577a99eafa7b4bdfa90265e862bdc5cd, State: Initialized, Role: FOLLOWER
I20260620 07:06:51.944365 10622 consensus_queue.cc:260] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:51.944618 10622 ts_tablet_manager.cc:1428] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:06:52.081461 10605 raft_consensus.cc:482] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:06:52.081604 10605 raft_consensus.cc:504] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:52.081991 10605 leader_election.cc:288] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:52.082386 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d08c0332a2345aba386be92b1c5d50d" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" is_pre_election: true
I20260620 07:06:52.082432 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d08c0332a2345aba386be92b1c5d50d" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be" is_pre_election: true
I20260620 07:06:52.082527 10174 raft_consensus.cc:2454] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:52.082562 10445 raft_consensus.cc:2454] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 0.
I20260620 07:06:52.082880 10246 leader_election.cc:302] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:52.083045 10605 raft_consensus.cc:2781] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:06:52.083102 10605 raft_consensus.cc:482] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:06:52.083139 10605 raft_consensus.cc:3037] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:52.084144 10605 raft_consensus.cc:504] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:52.084460 10605 leader_election.cc:288] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Requested vote from peers 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503), 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911)
I20260620 07:06:52.084797 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d08c0332a2345aba386be92b1c5d50d" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b319d3f7ce4e59a103966a74cf20be"
I20260620 07:06:52.084825 10174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d08c0332a2345aba386be92b1c5d50d" candidate_uuid: "7e45032335974805b36fc5c34a6393c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "577a99eafa7b4bdfa90265e862bdc5cd"
I20260620 07:06:52.084899 10445 raft_consensus.cc:3037] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:52.084928 10174 raft_consensus.cc:3037] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:52.085731 10445 raft_consensus.cc:2454] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:52.085819 10174 raft_consensus.cc:2454] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45032335974805b36fc5c34a6393c6 in term 1.
I20260620 07:06:52.085973 10246 leader_election.cc:302] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b319d3f7ce4e59a103966a74cf20be, 7e45032335974805b36fc5c34a6393c6; no voters:
I20260620 07:06:52.086124 10605 raft_consensus.cc:2781] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:52.086215 10605 raft_consensus.cc:686] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7e45032335974805b36fc5c34a6393c6, State: Running, Role: LEADER
I20260620 07:06:52.086346 10605 consensus_queue.cc:237] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } }
I20260620 07:06:52.087800 10014 catalog_manager.cc:5495] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e45032335974805b36fc5c34a6393c6 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "7e45032335974805b36fc5c34a6393c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e45032335974805b36fc5c34a6393c6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 } health_report { overall_health: UNKNOWN } } }
I20260620 07:06:52.136586 10445 raft_consensus.cc:1261] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:52.136611 10174 raft_consensus.cc:1261] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd [term 1 FOLLOWER]: Refusing update from remote peer 7e45032335974805b36fc5c34a6393c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:06:52.136916 10605 consensus_queue.cc:1035] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b319d3f7ce4e59a103966a74cf20be" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 32911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:06:52.137058 10586 consensus_queue.cc:1035] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "577a99eafa7b4bdfa90265e862bdc5cd" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 07:06:52.147851 10593 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 07:06:53.691864 10017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:53.692011 10017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:06:53.694084 10017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4d569ec7c2541d9b88e2e63b6d8c456: Sending DeleteTablet for 3 replicas of tablet 6d08c0332a2345aba386be92b1c5d50d
I20260620 07:06:53.694541 10154 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d08c0332a2345aba386be92b1c5d50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:53 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36078
I20260620 07:06:53.694631 10421 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d08c0332a2345aba386be92b1c5d50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:53 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39028
I20260620 07:06:53.694671 10294 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d08c0332a2345aba386be92b1c5d50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:06:53 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260620 07:06:53.694973 10665 tablet_replica.cc:330] T 6d08c0332a2345aba386be92b1c5d50d P 577a99eafa7b4bdfa90265e862bdc5cd: stopping tablet replica
I20260620 07:06:53.694978 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10089
I20260620 07:06:53.695721 10666 tablet_replica.cc:330] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: stopping tablet replica
I20260620 07:06:53.695829 10666 raft_consensus.cc:2229] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:06:53.695920 10666 raft_consensus.cc:2258] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:53.696102 10667 tablet_replica.cc:330] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: stopping tablet replica
I20260620 07:06:53.696208 10667 raft_consensus.cc:2229] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:06:53.696372 10667 raft_consensus.cc:2258] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:06:53.696995 10666 ts_tablet_manager.cc:1905] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:53.698935 10666 ts_tablet_manager.cc:1918] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 07:06:53.698984 10666 log.cc:1196] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/6d08c0332a2345aba386be92b1c5d50d
I20260620 07:06:53.699146 10666 ts_tablet_manager.cc:1939] T 6d08c0332a2345aba386be92b1c5d50d P 63b319d3f7ce4e59a103966a74cf20be: Deleting consensus metadata
I20260620 07:06:53.699774 10667 ts_tablet_manager.cc:1905] T 6d08c0332a2345aba386be92b1c5d50d P 7e45032335974805b36fc5c34a6393c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:06:53.699878 10002 catalog_manager.cc:4841] TS 63b319d3f7ce4e59a103966a74cf20be (127.8.24.3:32911): tablet 6d08c0332a2345aba386be92b1c5d50d (table test-table [id=08d20168346748bea9bd1d31ff115785]) successfully deleted
W20260620 07:06:53.700941 10002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:34503 (error 108)
I20260620 07:06:53.701172 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10229
I20260620 07:06:53.702476 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:53.702089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34503 (local address 127.0.0.1:36094)
0620 07:06:53.702307 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:53.702378 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:34503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
W20260620 07:06:53.702596 10002 catalog_manager.cc:4584] TS 577a99eafa7b4bdfa90265e862bdc5cd (127.8.24.1:34503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d08c0332a2345aba386be92b1c5d50d: Network error: Client connection negotiation failed: client connection to 127.8.24.1:34503: connect: Connection refused (error 111)
I20260620 07:06:53.706961 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10369
I20260620 07:06:53.707554 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:53.706886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43601 (local address 127.0.0.1:44518)
0620 07:06:53.706999 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:53.707049 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 07:06:53.712078 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 9985
2026-06-20T07:06:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6216 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:53 dist-test-slave-wjw3 krb5kdc[10672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:53 dist-test-slave-wjw3 krb5kdc[10672](info): set up 2 sockets
Jun 20 07:06:53 dist-test-slave-wjw3 krb5kdc[10672](info): commencing operation
krb5kdc: starting...
W20260620 07:06:54.667192 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:54 dist-test-slave-wjw3 krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:54.775808 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35657
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:54.879622 10685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:54.879868 10685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:54.879921 10685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:54.883824 10685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:54.883879 10685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:54.883905 10685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:54.883929 10685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:54.883955 10685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:54.889007 10685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:54.890017 10685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:54.891017 10685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:54.899586 10693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:54.899586 10690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:54.899682 10691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:54.899752 10685 server_base.cc:1034] running on GCE node
I20260620 07:06:54.900482 10685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:54.901582 10685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:54.902756 10685 hybrid_clock.cc:648] HybridClock initialized: now 1781939214902739 us; error 33 us; skew 500 ppm
Jun 20 07:06:54 dist-test-slave-wjw3 krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939214, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:54.905417 10685 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:54.906311 10685 webserver.cc:460] Webserver started at http://127.8.24.62:43713/ using document root <none> and password file <none>
I20260620 07:06:54.906826 10685 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:54.906890 10685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:54.907131 10685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:54.909938 10685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "1096a75cd5524dd08a96e04f42a0d61c"
format_stamp: "Formatted at 2026-06-20 07:06:54 on dist-test-slave-wjw3"
server_key: "b40486e29b1df9917b421c9fc2440aa6"
server_key_iv: "b1e6edf7f3f91128a0a8acf42984a04c"
server_key_version: "encryptionkey@0"
I20260620 07:06:54.910501 10685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "1096a75cd5524dd08a96e04f42a0d61c"
format_stamp: "Formatted at 2026-06-20 07:06:54 on dist-test-slave-wjw3"
server_key: "b40486e29b1df9917b421c9fc2440aa6"
server_key_iv: "b1e6edf7f3f91128a0a8acf42984a04c"
server_key_version: "encryptionkey@0"
I20260620 07:06:54.913990 10685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:06:54.916242 10700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:54.917407 10685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260620 07:06:54.917513 10685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "1096a75cd5524dd08a96e04f42a0d61c"
format_stamp: "Formatted at 2026-06-20 07:06:54 on dist-test-slave-wjw3"
server_key: "b40486e29b1df9917b421c9fc2440aa6"
server_key_iv: "b1e6edf7f3f91128a0a8acf42984a04c"
server_key_version: "encryptionkey@0"
I20260620 07:06:54.917600 10685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:54.929430 10685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:54.931998 10685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:54.932160 10685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:54.938431 10685 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35657
I20260620 07:06:54.938445 10752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35657 every 8 connection(s)
I20260620 07:06:54.939312 10685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:54.940424 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10685
I20260620 07:06:54.940531 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:54.940768 8288 external_mini_cluster.cc:1428] Setting key 9e2eacc8b137d3bb516836b5e86e208c
I20260620 07:06:54.942241 10753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:54.948418 10753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: Bootstrap starting.
Jun 20 07:06:54 dist-test-slave-wjw3 krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:54.950719 10753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:54.951406 10753 log.cc:824] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:54.953464 10753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: No bootstrap required, opened a new log
I20260620 07:06:54.954020 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:54.941990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.8.24.62:35657)
0620 07:06:54.942365 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:54.942380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:54.942414 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:06:54.943035 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:54.943051 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:54.943371 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:54.943690 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:54.944510 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:54.945488 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:54.946576 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:54.947022 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:54.947127 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:54.949671 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:54.949675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:54.949688 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:54.949715 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:54.951273 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:54.951725 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:54.951730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:54.951735 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:54.951874 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:54.952141 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:54.952145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:54.952146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:54.952531 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:54.952669 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:54.952939 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 07:06:54.953209 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260620 07:06:54.956910 10753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } }
I20260620 07:06:54.957130 10753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:54.957185 10753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096a75cd5524dd08a96e04f42a0d61c, State: Initialized, Role: FOLLOWER
I20260620 07:06:54.957576 10753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } }
I20260620 07:06:54.957702 10753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:54.957767 10753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:54.957863 10753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:54.958720 10753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } }
I20260620 07:06:54.959033 10753 leader_election.cc:302] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1096a75cd5524dd08a96e04f42a0d61c; no voters:
I20260620 07:06:54.959308 10753 leader_election.cc:288] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:54.959405 10758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:54.959656 10758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [term 1 LEADER]: Becoming Leader. State: Replica: 1096a75cd5524dd08a96e04f42a0d61c, State: Running, Role: LEADER
I20260620 07:06:54.959995 10758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } }
I20260620 07:06:54.960451 10753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:54.961890 10760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1096a75cd5524dd08a96e04f42a0d61c. Latest consensus state: current_term: 1 leader_uuid: "1096a75cd5524dd08a96e04f42a0d61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } } }
I20260620 07:06:54.962018 10760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:54.962286 10759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1096a75cd5524dd08a96e04f42a0d61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1096a75cd5524dd08a96e04f42a0d61c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35657 } } }
I20260620 07:06:54.962369 10759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c [sys.catalog]: This master's current role is: LEADER
W20260620 07:06:54.964922 10773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:06:54.964993 10773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:06:54.965062 10772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:06:54.965775 10772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:54.971086 10772 catalog_manager.cc:1347] Generated new cluster ID: 82ca157b0ec8456fa674c7a1f09f4545
I20260620 07:06:54.971146 10772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:54.979831 10772 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:54.980845 10772 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:54.984772 10772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1096a75cd5524dd08a96e04f42a0d61c: Generated new TSK 0
I20260620 07:06:54.985543 10772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:06:55.009989 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:55.118120 10779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:55.118369 10779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:55.118410 10779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:55.122387 10779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:55.122447 10779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:55.122503 10779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:06:55.127684 10779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:55.128679 10779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:55.129663 10779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:55.138648 10784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:55.138659 10785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:55.138788 10779 server_base.cc:1034] running on GCE node
W20260620 07:06:55.138659 10787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:55.139266 10779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:55.139885 10779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:55.141031 10779 hybrid_clock.cc:648] HybridClock initialized: now 1781939215141014 us; error 29 us; skew 500 ppm
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:55.143492 10779 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:06:55.144264 10779 webserver.cc:460] Webserver started at http://127.8.24.1:41031/ using document root <none> and password file <none>
I20260620 07:06:55.144760 10779 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:55.144821 10779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:55.145007 10779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:55.146883 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "d22f6f723483464ab6ceb27b209c92c2"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "edb17cfc9f456134104ed85a1f38738c"
server_key_iv: "8de804133a2c039dbc3bac5244a00cc8"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.147403 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "d22f6f723483464ab6ceb27b209c92c2"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "edb17cfc9f456134104ed85a1f38738c"
server_key_iv: "8de804133a2c039dbc3bac5244a00cc8"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.150695 10779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:06:55.152853 10794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.153924 10779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:55.154031 10779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "d22f6f723483464ab6ceb27b209c92c2"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "edb17cfc9f456134104ed85a1f38738c"
server_key_iv: "8de804133a2c039dbc3bac5244a00cc8"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.154110 10779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:55.193040 10779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:55.196060 10779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:55.196224 10779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:55.196769 10779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:55.197551 10779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:55.197595 10779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.197644 10779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:55.197670 10779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.205530 10779 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45451
I20260620 07:06:55.205581 10907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45451 every 8 connection(s)
I20260620 07:06:55.206374 10779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:55.215912 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10779
I20260620 07:06:55.216023 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:06:55.216284 8288 external_mini_cluster.cc:1428] Setting key c79b56d6b56f4b1e3a64f270351259a6
I20260620 07:06:55.218858 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.208068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34875 (local address 127.8.24.62:35657)
0620 07:06:55.208236 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:55.208240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:55.209045 (+ 805us) server_negotiation.cc:407] Connection header received
0620 07:06:55.210213 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:55.210215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:55.210262 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:55.210330 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:55.211407 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.211957 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.212985 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.213198 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.213262 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.215490 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:55.215494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:55.215496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:55.215521 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:55.216941 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.217375 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.217379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.217381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.217465 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.217824 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.217826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.217827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.218004 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:55.218075 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:55.218643 (+ 568us) server_negotiation.cc:299] Negotiation successful
0620 07:06:55.218742 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 07:06:55.219866 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.208471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35657 (local address 127.8.24.1:34875)
0620 07:06:55.208898 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:55.208939 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:55.210026 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:55.210480 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:55.210496 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:55.210835 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:55.211299 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.211313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.212043 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.212045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.212896 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.212904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.213299 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.213301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.213433 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.213905 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:55.213920 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:55.215295 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:55.217038 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.217044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.217059 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.217287 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.217560 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.217563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.217565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.217738 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.218099 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:55.218105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:55.218410 (+ 305us) client_negotiation.cc:769] Sending connection context
0620 07:06:55.218613 (+ 203us) client_negotiation.cc:240] Negotiation successful
0620 07:06:55.218934 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260620 07:06:55.220963 10908 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35657
I20260620 07:06:55.221175 10908 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:55.221673 10908 heartbeater.cc:511] Master 127.8.24.62:35657 requested a full tablet report, sending...
I20260620 07:06:55.223050 10717 ts_manager.cc:194] Registered new tserver with Master: d22f6f723483464ab6ceb27b209c92c2 (127.8.24.1:45451)
I20260620 07:06:55.224128 10717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:34875
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:06:55.239997 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:55.346704 10913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:55.346984 10913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:55.347045 10913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:55.351105 10913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:55.351168 10913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:55.351244 10913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:06:55.356457 10913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:55.357462 10913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:55.358461 10913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:55.367724 10919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:55.367779 10921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:55.367743 10918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:55.368027 10913 server_base.cc:1034] running on GCE node
I20260620 07:06:55.368400 10913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:55.368973 10913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:55.370122 10913 hybrid_clock.cc:648] HybridClock initialized: now 1781939215370105 us; error 34 us; skew 500 ppm
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:55.372624 10913 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:06:55.373350 10913 webserver.cc:460] Webserver started at http://127.8.24.2:40101/ using document root <none> and password file <none>
I20260620 07:06:55.373833 10913 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:55.373885 10913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:55.374054 10913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:55.376009 10913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "dac18182bd5b45cab33f4c6c72954f7f"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "c11ab9217ced887f06eed0bef25ac27d"
server_key_iv: "ff5c8316c54cb2f7efff65e68ab2fa46"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.376497 10913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "dac18182bd5b45cab33f4c6c72954f7f"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "c11ab9217ced887f06eed0bef25ac27d"
server_key_iv: "ff5c8316c54cb2f7efff65e68ab2fa46"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.379992 10913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:06:55.382149 10928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.383237 10913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:55.383327 10913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "dac18182bd5b45cab33f4c6c72954f7f"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "c11ab9217ced887f06eed0bef25ac27d"
server_key_iv: "ff5c8316c54cb2f7efff65e68ab2fa46"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.383419 10913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:55.402387 10913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:55.405112 10913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:55.405274 10913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:55.405838 10913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:55.406620 10913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:55.406664 10913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.406708 10913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:55.406735 10913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.414621 10913 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:34961
I20260620 07:06:55.414665 11041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:34961 every 8 connection(s)
I20260620 07:06:55.415459 10913 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:55.424901 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 10913
I20260620 07:06:55.424996 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:06:55.425246 8288 external_mini_cluster.cc:1428] Setting key eb30930b56c7a2552cc4fa94d870e857
I20260620 07:06:55.427943 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.417222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45259 (local address 127.8.24.62:35657)
0620 07:06:55.417341 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:55.417345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:55.418302 (+ 957us) server_negotiation.cc:407] Connection header received
0620 07:06:55.419140 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:55.419143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:55.419197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:55.419269 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:55.420393 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.420947 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.422066 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.422294 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.422380 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.424568 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:55.424571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:55.424573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:55.424599 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:55.426018 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.426498 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.426502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.426504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.426592 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.426894 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.426897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.426899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.427074 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:55.427156 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:55.427673 (+ 517us) server_negotiation.cc:299] Negotiation successful
0620 07:06:55.427792 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:06:55.428810 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.417576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35657 (local address 127.8.24.2:45259)
0620 07:06:55.418172 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:55.418210 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:55.418944 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:55.419427 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:55.419460 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:55.419794 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:55.420262 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.420274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.421079 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.421081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.421959 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.421969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.422423 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.422428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.422588 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.423051 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:55.423066 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:55.424405 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:55.426117 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.426121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.426133 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.426393 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.426671 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.426675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.426677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.426812 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.427164 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:55.427171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:55.427410 (+ 239us) client_negotiation.cc:769] Sending connection context
0620 07:06:55.427623 (+ 213us) client_negotiation.cc:240] Negotiation successful
0620 07:06:55.427938 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":110,"threads_started":1}
I20260620 07:06:55.429822 11042 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35657
I20260620 07:06:55.430042 11042 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:55.430480 11042 heartbeater.cc:511] Master 127.8.24.62:35657 requested a full tablet report, sending...
I20260620 07:06:55.431412 10717 ts_manager.cc:194] Registered new tserver with Master: dac18182bd5b45cab33f4c6c72954f7f (127.8.24.2:34961)
I20260620 07:06:55.431959 10717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:45259
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:06:55.448382 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:55.553625 11047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:55.553877 11047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:55.553916 11047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:55.557972 11047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:55.558033 11047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:55.558097 11047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:06:55.563374 11047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:55.564419 11047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:55.565342 11047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:55.574204 11055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:55.574219 11053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:55.574313 11052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:55.574504 11047 server_base.cc:1034] running on GCE node
I20260620 07:06:55.574936 11047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:55.575572 11047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:55.576726 11047 hybrid_clock.cc:648] HybridClock initialized: now 1781939215576707 us; error 33 us; skew 500 ppm
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:55.579241 11047 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:06:55.580150 11047 webserver.cc:460] Webserver started at http://127.8.24.3:45969/ using document root <none> and password file <none>
I20260620 07:06:55.580657 11047 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:55.580720 11047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:55.580896 11047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:55.582857 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "24f99cee1b76458eb3c548d3f5dad8b3"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "3aca44b4efdf2ca5117563103ceb7fea"
server_key_iv: "c939da648cfbda09aab125950479cf82"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.583396 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "24f99cee1b76458eb3c548d3f5dad8b3"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "3aca44b4efdf2ca5117563103ceb7fea"
server_key_iv: "c939da648cfbda09aab125950479cf82"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.586866 11047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:06:55.589166 11062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.590279 11047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:06:55.590385 11047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "24f99cee1b76458eb3c548d3f5dad8b3"
format_stamp: "Formatted at 2026-06-20 07:06:55 on dist-test-slave-wjw3"
server_key: "3aca44b4efdf2ca5117563103ceb7fea"
server_key_iv: "c939da648cfbda09aab125950479cf82"
server_key_version: "encryptionkey@0"
I20260620 07:06:55.590472 11047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:55.604018 11047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:55.606536 11047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:55.606694 11047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:55.607239 11047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:06:55.608026 11047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:06:55.608071 11047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.608120 11047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:06:55.608150 11047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:55.616003 11047 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46355
I20260620 07:06:55.616025 11175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46355 every 8 connection(s)
I20260620 07:06:55.616789 11047 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:06:55.623642 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11047
I20260620 07:06:55.623749 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:06:55.624001 8288 external_mini_cluster.cc:1428] Setting key 10e06e9ec5f5068f3b5f493a16c155c0
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[10672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939215, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:55.628734 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.618393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47379 (local address 127.8.24.62:35657)
0620 07:06:55.618521 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:55.618525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:55.619469 (+ 944us) server_negotiation.cc:407] Connection header received
0620 07:06:55.620315 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:55.620318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:55.620388 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:55.620459 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:55.621572 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.622098 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.623180 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.623400 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.623517 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.625801 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:55.625804 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:55.625806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:55.625830 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:55.626902 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.627340 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.627344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.627346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.627419 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:55.627732 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:55.627735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:55.627737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:55.627911 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:55.628001 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:55.628463 (+ 462us) server_negotiation.cc:299] Negotiation successful
0620 07:06:55.628582 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:06:55.629499 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:55.618751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35657 (local address 127.8.24.3:47379)
0620 07:06:55.619332 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0620 07:06:55.619370 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:06:55.620128 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:06:55.620646 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:06:55.620653 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:06:55.620972 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:06:55.621436 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.621454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.622208 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.622211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.623070 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:06:55.623078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:06:55.623559 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:06:55.623563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:06:55.623717 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:55.624303 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:06:55.624319 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:06:55.625652 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:06:55.627000 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.627005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.627021 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.627252 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.627521 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:06:55.627524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:06:55.627527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:06:55.627650 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:06:55.628001 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:06:55.628008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:06:55.628246 (+ 238us) client_negotiation.cc:769] Sending connection context
0620 07:06:55.628410 (+ 164us) client_negotiation.cc:240] Negotiation successful
0620 07:06:55.628722 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":103,"threads_started":1}
I20260620 07:06:55.630467 11176 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35657
I20260620 07:06:55.630684 11176 heartbeater.cc:464] Registering TS with master...
I20260620 07:06:55.631125 11176 heartbeater.cc:511] Master 127.8.24.62:35657 requested a full tablet report, sending...
I20260620 07:06:55.632114 10717 ts_manager.cc:194] Registered new tserver with Master: 24f99cee1b76458eb3c548d3f5dad8b3 (127.8.24.3:46355)
I20260620 07:06:55.632640 10717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:47379
I20260620 07:06:55.637324 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 07:06:55.649068 10756 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:06:55.647847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.8.24.62:35657)
0620 07:06:55.647986 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:55.647991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:55.648117 (+ 126us) server_negotiation.cc:407] Connection header received
0620 07:06:55.648218 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:55.648220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:55.648260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:55.648322 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:55.648936 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45034: BlockingRecv error: recv got EOF from 127.0.0.1:45034 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:06:55.650656 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10779
I20260620 07:06:55.655200 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10913
I20260620 07:06:55.659847 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11047
I20260620 07:06:55.664222 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 10685
2026-06-20T07:06:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1948 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[11188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[11188](info): set up 2 sockets
Jun 20 07:06:55 dist-test-slave-wjw3 krb5kdc[11188](info): commencing operation
krb5kdc: starting...
W20260620 07:06:56.607097 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:06:56 dist-test-slave-wjw3 krb5kdc[11188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:06:56.714175 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35411
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:06:56.817279 11201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:06:56.817545 11201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:06:56.817584 11201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:06:56.821547 11201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:06:56.821604 11201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:06:56.821622 11201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:06:56.821636 11201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:06:56.821655 11201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:06:56.826764 11201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:06:56.827757 11201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:06:56.828687 11201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:06:56.836310 11206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:56.836375 11201 server_base.cc:1034] running on GCE node
W20260620 07:06:56.836324 11207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:06:56.836383 11209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:06:56.837029 11201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:06:56.838073 11201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:06:56.839229 11201 hybrid_clock.cc:648] HybridClock initialized: now 1781939216839215 us; error 25 us; skew 500 ppm
Jun 20 07:06:56 dist-test-slave-wjw3 krb5kdc[11188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939216, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:06:56.844504 11201 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:06:56.845320 11201 webserver.cc:460] Webserver started at http://127.8.24.62:34153/ using document root <none> and password file <none>
I20260620 07:06:56.845816 11201 fs_manager.cc:359] Metadata directory not provided
I20260620 07:06:56.845877 11201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:06:56.846089 11201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:06:56.848076 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "54f041b8e28547999f17ecfec7ae37c5"
format_stamp: "Formatted at 2026-06-20 07:06:56 on dist-test-slave-wjw3"
server_key: "dabd3b18268a060cf943c2eb46aa98bc"
server_key_iv: "182658c778c122830281e8270d735597"
server_key_version: "encryptionkey@0"
I20260620 07:06:56.848615 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "54f041b8e28547999f17ecfec7ae37c5"
format_stamp: "Formatted at 2026-06-20 07:06:56 on dist-test-slave-wjw3"
server_key: "dabd3b18268a060cf943c2eb46aa98bc"
server_key_iv: "182658c778c122830281e8270d735597"
server_key_version: "encryptionkey@0"
I20260620 07:06:56.852201 11201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:06:56.854436 11216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:06:56.855616 11201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:06:56.855721 11201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "54f041b8e28547999f17ecfec7ae37c5"
format_stamp: "Formatted at 2026-06-20 07:06:56 on dist-test-slave-wjw3"
server_key: "dabd3b18268a060cf943c2eb46aa98bc"
server_key_iv: "182658c778c122830281e8270d735597"
server_key_version: "encryptionkey@0"
I20260620 07:06:56.855811 11201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:06:56.870534 11201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:06:56.873464 11201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:06:56.873629 11201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:06:56.879951 11201 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35411
I20260620 07:06:56.880235 11268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35411 every 8 connection(s)
I20260620 07:06:56.880934 11201 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:06:56.883530 11269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:06:56.888885 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11201
I20260620 07:06:56.888978 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:06:56.889112 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: Bootstrap starting.
I20260620 07:06:56.889213 8288 external_mini_cluster.cc:1428] Setting key f09711320ca02c26d369e8c16c80b296
I20260620 07:06:56.891500 11269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: Neither blocks nor log segments found. Creating new log.
I20260620 07:06:56.892179 11269 log.cc:824] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: Log is configured to *not* fsync() on all Append() calls
I20260620 07:06:56.893819 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: No bootstrap required, opened a new log
Jun 20 07:06:56 dist-test-slave-wjw3 krb5kdc[11188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:06:56.897177 11269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } }
I20260620 07:06:56.897446 11269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:06:56.897505 11269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f041b8e28547999f17ecfec7ae37c5, State: Initialized, Role: FOLLOWER
I20260620 07:06:56.897953 11269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } }
I20260620 07:06:56.898080 11269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:06:56.898135 11269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:06:56.898226 11269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:06:56.899230 11269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } }
I20260620 07:06:56.899588 11269 leader_election.cc:302] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f041b8e28547999f17ecfec7ae37c5; no voters:
I20260620 07:06:56.899899 11269 leader_election.cc:288] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:06:56.900004 11274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:06:56.900205 11274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [term 1 LEADER]: Becoming Leader. State: Replica: 54f041b8e28547999f17ecfec7ae37c5, State: Running, Role: LEADER
I20260620 07:06:56.900580 11274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } }
I20260620 07:06:56.901073 11269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:06:56.902838 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:56.890421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.8.24.62:35411)
0620 07:06:56.890842 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:56.890856 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:56.890891 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:06:56.891540 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:56.891558 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:56.891909 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:56.892209 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:56.892986 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:56.893917 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:56.894766 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:56.895230 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:56.895369 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:56.897874 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:56.897878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:56.897890 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:06:56.897915 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:56.899752 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:56.900236 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:56.900242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:56.900248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:56.900384 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:06:56.900673 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:06:56.900676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:06:56.900678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:06:56.901157 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:56.901344 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:56.901638 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 07:06:56.901921 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260620 07:06:56.904193 11276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f041b8e28547999f17ecfec7ae37c5. Latest consensus state: current_term: 1 leader_uuid: "54f041b8e28547999f17ecfec7ae37c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } } }
I20260620 07:06:56.905025 11276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:56.904173 11275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54f041b8e28547999f17ecfec7ae37c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f041b8e28547999f17ecfec7ae37c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35411 } } }
I20260620 07:06:56.905314 11275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:06:56.905844 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 07:06:56.906313 11288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:06:56.906387 11288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:06:56.913944 11291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:06:56.915527 11291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:06:56.922866 11291 catalog_manager.cc:1347] Generated new cluster ID: 95d169a3dc9046dfb80059ecea2b60ea
I20260620 07:06:56.922925 11291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:06:56.923502 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:06:56.918112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.8.24.62:35411)
0620 07:06:56.918214 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 07:06:56.918218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:06:56.918449 (+ 231us) server_negotiation.cc:407] Connection header received
0620 07:06:56.918639 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:06:56.918642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:06:56.918686 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:06:56.918735 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:06:56.919385 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:56.920203 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:56.921370 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:06:56.921669 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:06:56.921765 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:06:56.922522 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:06:56.922526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:06:56.922528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:06:56.923031 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:06:56.923063 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:06:56.923083 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:06:56.923169 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:06:56.923287 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 07:06:56.923346 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:06:56.969779 11291 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:06:56.970733 11291 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:06:56.977274 11291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54f041b8e28547999f17ecfec7ae37c5: Generated new TSK 0
I20260620 07:06:56.977944 11291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:06:56.982548 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11201
2026-06-20T07:06:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1316 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:06:57 dist-test-slave-wjw3 krb5kdc[11302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:06:57 dist-test-slave-wjw3 krb5kdc[11302](info): set up 2 sockets
Jun 20 07:06:57 dist-test-slave-wjw3 krb5kdc[11302](info): commencing operation
krb5kdc: starting...
W20260620 07:06:57.960525 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260620 07:06:58.202960 8288 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.242s user 0.000s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260620 07:06:58.390632 8288 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.188s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260620 07:06:58.633476 8288 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.243s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/test-user.keytab.
W20260620 07:06:59.528091 8288 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.894s user 0.000s sys 0.002s
Jun 20 07:06:59 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:06:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
W20260620 07:07:01.069442 8288 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.24.62: real 1.521s user 0.001s sys 0.001s
I20260620 07:07:01.069963 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39261
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:01.185014 11315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:01.185267 11315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:01.185304 11315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:01.189419 11315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:01.189473 11315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:01.189492 11315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:01.189518 11315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:01.189546 11315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:01.194789 11315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:01.195890 11315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:01.196909 11315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:01.205085 11320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.205102 11321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.205216 11315 server_base.cc:1034] running on GCE node
W20260620 07:07:01.205073 11323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.205888 11315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:01.206959 11315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:01.208112 11315 hybrid_clock.cc:648] HybridClock initialized: now 1781939221208093 us; error 31 us; skew 500 ppm
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:01.210582 11315 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:01.211391 11315 webserver.cc:460] Webserver started at http://127.8.24.62:34761/ using document root <none> and password file <none>
I20260620 07:07:01.211948 11315 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:01.212013 11315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:01.212181 11315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:01.214143 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "c10baca430a4465886d0314df36a54e4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "f1d48d5ff3b82d1e4812cabeebe00286"
server_key_iv: "007cc32c1926c13301aa32dd5521e854"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.214658 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "c10baca430a4465886d0314df36a54e4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "f1d48d5ff3b82d1e4812cabeebe00286"
server_key_iv: "007cc32c1926c13301aa32dd5521e854"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.218307 11315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:07:01.220649 11330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.221968 11315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:01.222100 11315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "c10baca430a4465886d0314df36a54e4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "f1d48d5ff3b82d1e4812cabeebe00286"
server_key_iv: "007cc32c1926c13301aa32dd5521e854"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.222206 11315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:01.234059 11315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:01.236644 11315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:01.236799 11315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:01.242936 11315 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39261
I20260620 07:07:01.243170 11382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39261 every 8 connection(s)
I20260620 07:07:01.243856 11315 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:01.245357 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11315
I20260620 07:07:01.245447 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:01.245714 8288 external_mini_cluster.cc:1428] Setting key dbfea775d99207346238e094c1ca28ac
I20260620 07:07:01.246894 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:01.253583 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4: Bootstrap starting.
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:01.255976 11383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:01.256870 11383 log.cc:824] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:01.258982 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4: No bootstrap required, opened a new log
I20260620 07:07:01.259759 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.247124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.8.24.62:39261)
0620 07:07:01.247529 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:01.247549 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:01.247590 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:07:01.248392 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:01.248413 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:01.248818 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:01.249179 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:01.249776 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.251247 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.252131 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.252849 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.253000 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.255224 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:01.255231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:01.255245 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:01.255275 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:01.257073 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.257521 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.257527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.257533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.257671 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.257926 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.257928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.257930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.258310 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:01.258443 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:01.258708 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 07:07:01.258958 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260620 07:07:01.262423 11383 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } }
I20260620 07:07:01.262635 11383 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:01.262686 11383 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10baca430a4465886d0314df36a54e4, State: Initialized, Role: FOLLOWER
I20260620 07:07:01.263082 11383 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } }
I20260620 07:07:01.263199 11383 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:01.263258 11383 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:01.263350 11383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:01.264178 11383 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } }
I20260620 07:07:01.264492 11383 leader_election.cc:302] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10baca430a4465886d0314df36a54e4; no voters:
I20260620 07:07:01.264756 11383 leader_election.cc:288] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:01.264869 11388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:01.265054 11388 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [term 1 LEADER]: Becoming Leader. State: Replica: c10baca430a4465886d0314df36a54e4, State: Running, Role: LEADER
I20260620 07:07:01.265370 11388 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } }
I20260620 07:07:01.265857 11383 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:01.267045 11390 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10baca430a4465886d0314df36a54e4. Latest consensus state: current_term: 1 leader_uuid: "c10baca430a4465886d0314df36a54e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } } }
I20260620 07:07:01.267169 11390 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:01.267021 11389 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10baca430a4465886d0314df36a54e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10baca430a4465886d0314df36a54e4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39261 } } }
I20260620 07:07:01.267412 11389 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:01.267607 11397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:01.270399 11397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:01.275385 11397 catalog_manager.cc:1347] Generated new cluster ID: df87083d0e8b4d709c5bcd6e7d387d67
I20260620 07:07:01.275465 11397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:01.287629 11397 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:01.288797 11397 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:01.307874 11397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10baca430a4465886d0314df36a54e4: Generated new TSK 0
I20260620 07:07:01.308615 11397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:01.340008 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:01.444912 11409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:01.445143 11409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:01.445181 11409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:01.449028 11409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:01.449077 11409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:01.449132 11409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:01.454070 11409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:01.455049 11409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:01.456033 11409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:01.464774 11414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.464771 11415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.464911 11417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.465070 11409 server_base.cc:1034] running on GCE node
I20260620 07:07:01.465396 11409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:01.465952 11409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:01.467100 11409 hybrid_clock.cc:648] HybridClock initialized: now 1781939221467078 us; error 38 us; skew 500 ppm
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:01.469642 11409 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:01.470410 11409 webserver.cc:460] Webserver started at http://127.8.24.1:41471/ using document root <none> and password file <none>
I20260620 07:07:01.470906 11409 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:01.470968 11409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:01.471146 11409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:01.473065 11409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "f628a193244840fbad4d0d65a4296a28"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "8a1d7e057bab4a0a141402cc119e569c"
server_key_iv: "5decd74f39f048aa795f6878636c5948"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.473572 11409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "f628a193244840fbad4d0d65a4296a28"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "8a1d7e057bab4a0a141402cc119e569c"
server_key_iv: "5decd74f39f048aa795f6878636c5948"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.476989 11409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:01.479080 11424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.480228 11409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:01.480327 11409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "f628a193244840fbad4d0d65a4296a28"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "8a1d7e057bab4a0a141402cc119e569c"
server_key_iv: "5decd74f39f048aa795f6878636c5948"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.480412 11409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:01.500339 11409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:01.503252 11409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:01.503409 11409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:01.503949 11409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:01.504701 11409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:01.504746 11409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.504796 11409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:01.504824 11409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.512709 11409 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:39769
I20260620 07:07:01.512737 11537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:39769 every 8 connection(s)
I20260620 07:07:01.513494 11409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:01.515334 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11409
I20260620 07:07:01.515450 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:01.515710 8288 external_mini_cluster.cc:1428] Setting key a037542f518160203e3e28e63bb47cb6
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:01.526227 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.515128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:36627 (local address 127.8.24.62:39261)
0620 07:07:01.515265 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:01.515269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:01.516117 (+ 848us) server_negotiation.cc:407] Connection header received
0620 07:07:01.517478 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:01.517481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:01.517540 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:01.517607 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:01.518693 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.519218 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.520358 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.520598 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.520684 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.522881 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:01.522885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:01.522888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:01.522913 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:01.524226 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.524673 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.524676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.524678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.524755 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.525157 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.525161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.525163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.525370 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:01.525455 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:01.525915 (+ 460us) server_negotiation.cc:299] Negotiation successful
0620 07:07:01.526062 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:01.527100 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.515537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39261 (local address 127.8.24.1:36627)
0620 07:07:01.515968 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:01.516026 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:01.517285 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:01.517761 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:01.517768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:01.518105 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:01.518556 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.518569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.519333 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.519336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.520240 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.520250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.520730 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.520734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.520892 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.521408 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:01.521424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:01.522733 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:01.524329 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.524334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.524349 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.524584 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.524863 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.524866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.524868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.525075 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.525461 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:01.525467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:01.525701 (+ 234us) client_negotiation.cc:769] Sending connection context
0620 07:07:01.525860 (+ 159us) client_negotiation.cc:240] Negotiation successful
0620 07:07:01.526182 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":96,"threads_started":1}
I20260620 07:07:01.528297 11538 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39261
I20260620 07:07:01.528530 11538 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:01.528992 11538 heartbeater.cc:511] Master 127.8.24.62:39261 requested a full tablet report, sending...
I20260620 07:07:01.530479 11347 ts_manager.cc:194] Registered new tserver with Master: f628a193244840fbad4d0d65a4296a28 (127.8.24.1:39769)
I20260620 07:07:01.531682 11347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:36627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:01.541663 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:01.644490 11543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:01.644724 11543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:01.644762 11543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:01.648694 11543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:01.648739 11543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:01.648800 11543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:01.653712 11543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:01.654618 11543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:01.655553 11543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:07:01.664112 11543 server_base.cc:1034] running on GCE node
W20260620 07:07:01.664067 11549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.664065 11548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.664079 11551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.664613 11543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:01.665167 11543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:01.666314 11543 hybrid_clock.cc:648] HybridClock initialized: now 1781939221666300 us; error 29 us; skew 500 ppm
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:01.668889 11543 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:01.669725 11543 webserver.cc:460] Webserver started at http://127.8.24.2:43141/ using document root <none> and password file <none>
I20260620 07:07:01.670221 11543 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:01.670281 11543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:01.670451 11543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:01.672359 11543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "fb911b46858845f2b323cec0ec0364d4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "7abc850207036abe9b346e76f8d0e241"
server_key_iv: "73f5115751249a3821e90b608814eff3"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.672852 11543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "fb911b46858845f2b323cec0ec0364d4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "7abc850207036abe9b346e76f8d0e241"
server_key_iv: "73f5115751249a3821e90b608814eff3"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.676342 11543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:01.678393 11558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.679502 11543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:01.679594 11543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "fb911b46858845f2b323cec0ec0364d4"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "7abc850207036abe9b346e76f8d0e241"
server_key_iv: "73f5115751249a3821e90b608814eff3"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.679674 11543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:01.728535 11543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:01.731680 11543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:01.731885 11543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:01.732498 11543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:01.733278 11543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:01.733322 11543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.733372 11543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:01.733400 11543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.741340 11543 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:42143
I20260620 07:07:01.741357 11671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:42143 every 8 connection(s)
I20260620 07:07:01.742167 11543 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:01.747165 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11543
I20260620 07:07:01.747272 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:01.747573 8288 external_mini_cluster.cc:1428] Setting key 5096af282d294094b11e445cd2fac86b
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:01.755664 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.743823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52567 (local address 127.8.24.62:39261)
0620 07:07:01.743937 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:01.743940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:01.744740 (+ 800us) server_negotiation.cc:407] Connection header received
0620 07:07:01.746085 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:01.746088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:01.746143 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:01.746213 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:01.747378 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.747969 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.749356 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.749668 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.749760 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.752205 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:01.752209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:01.752212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:01.752238 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:01.753718 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.754178 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.754182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.754184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.754272 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.754579 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.754583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.754584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.754809 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:01.754905 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:01.755358 (+ 453us) server_negotiation.cc:299] Negotiation successful
0620 07:07:01.755497 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:07:01.756525 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.744194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39261 (local address 127.8.24.2:52567)
0620 07:07:01.744608 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:01.744648 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:01.745889 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:01.746393 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:01.746399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:01.746743 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:01.747195 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.747209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.748100 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.748102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.749238 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.749247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.749795 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.749797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.749932 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.750497 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:01.750516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:01.752049 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:01.753824 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.753829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.753841 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.754090 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.754371 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.754373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.754375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.754504 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.754892 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:01.754896 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:01.755131 (+ 235us) client_negotiation.cc:769] Sending connection context
0620 07:07:01.755285 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 07:07:01.755572 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260620 07:07:01.757628 11672 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39261
I20260620 07:07:01.757836 11672 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:01.758267 11672 heartbeater.cc:511] Master 127.8.24.62:39261 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:07:01.759245 11347 ts_manager.cc:194] Registered new tserver with Master: fb911b46858845f2b323cec0ec0364d4 (127.8.24.2:42143)
I20260620 07:07:01.759930 11347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:52567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:01.771941 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:01.874826 11677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:01.875066 11677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:01.875111 11677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:01.879019 11677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:01.879071 11677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:01.879138 11677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:01.884274 11677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:01.885211 11677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:01.886148 11677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:01.894883 11685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:01.894896 11682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.894927 11677 server_base.cc:1034] running on GCE node
W20260620 07:07:01.894894 11683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:01.895570 11677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:01.896127 11677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:01.897270 11677 hybrid_clock.cc:648] HybridClock initialized: now 1781939221897252 us; error 30 us; skew 500 ppm
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:01.902468 11677 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:01.903260 11677 webserver.cc:460] Webserver started at http://127.8.24.3:46517/ using document root <none> and password file <none>
I20260620 07:07:01.903784 11677 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:01.903841 11677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:01.904034 11677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:01.905920 11677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "2cfaea03a0a54c6b961836cb59400d53"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "83c591167a2982252884168aa85102b6"
server_key_iv: "855f8550c2109901d6710f0abe3640f8"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.906430 11677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "2cfaea03a0a54c6b961836cb59400d53"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "83c591167a2982252884168aa85102b6"
server_key_iv: "855f8550c2109901d6710f0abe3640f8"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.909788 11677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:01.911924 11692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.912988 11677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:01.913085 11677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "2cfaea03a0a54c6b961836cb59400d53"
format_stamp: "Formatted at 2026-06-20 07:07:01 on dist-test-slave-wjw3"
server_key: "83c591167a2982252884168aa85102b6"
server_key_iv: "855f8550c2109901d6710f0abe3640f8"
server_key_version: "encryptionkey@0"
I20260620 07:07:01.913173 11677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:01.959479 11677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:01.962507 11677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:01.962674 11677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:01.963207 11677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:01.963989 11677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:01.964030 11677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.964074 11677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:01.964102 11677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:01.972095 11677 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:45127
I20260620 07:07:01.972146 11805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:45127 every 8 connection(s)
I20260620 07:07:01.972909 11677 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:01.976852 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11677
I20260620 07:07:01.976956 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:01.977221 8288 external_mini_cluster.cc:1428] Setting key a9efbb3c5003a80f02ae3ca0827b289c
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:01.985256 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.974521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41353 (local address 127.8.24.62:39261)
0620 07:07:01.974657 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:01.974661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:01.975498 (+ 837us) server_negotiation.cc:407] Connection header received
0620 07:07:01.976856 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:01.976860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:01.976916 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:01.977003 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:01.978244 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.978976 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.980078 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.980290 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.980358 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.982463 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:01.982466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:01.982468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:01.982492 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:01.983680 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.984079 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.984082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.984083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.984143 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:01.984365 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:01.984367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:01.984368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:01.984509 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:01.984600 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:01.985055 (+ 455us) server_negotiation.cc:299] Negotiation successful
0620 07:07:01.985144 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:07:01.985982 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:01.974936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39261 (local address 127.8.24.3:41353)
0620 07:07:01.975342 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:01.975382 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:01.976658 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:01.977187 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:01.977194 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:01.977554 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:01.978121 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.978136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.979080 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.979083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.979968 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:01.979977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:01.980388 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:01.980390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:01.980520 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:01.981010 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:01.981025 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:01.982324 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:01.983774 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.983779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.983789 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.984012 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.984199 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:01.984201 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:01.984203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:01.984309 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:01.984648 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:01.984653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:01.984854 (+ 201us) client_negotiation.cc:769] Sending connection context
0620 07:07:01.984995 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 07:07:01.985215 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260620 07:07:01.986937 11806 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39261
I20260620 07:07:01.987123 11806 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:01.987553 11806 heartbeater.cc:511] Master 127.8.24.62:39261 requested a full tablet report, sending...
I20260620 07:07:01.988405 11347 ts_manager.cc:194] Registered new tserver with Master: 2cfaea03a0a54c6b961836cb59400d53 (127.8.24.3:45127)
I20260620 07:07:01.988931 11347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:41353
I20260620 07:07:01.990468 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:07:01 dist-test-slave-wjw3 krb5kdc[11302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 07:07:02 dist-test-slave-wjw3 krb5kdc[11302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939221, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:02.012739 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:02.005615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.8.24.62:39261)
0620 07:07:02.005727 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:02.005731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:02.005871 (+ 140us) server_negotiation.cc:407] Connection header received
0620 07:07:02.005972 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:02.005974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:02.006020 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:02.006085 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:02.006671 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:02.007158 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:02.007938 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:02.008140 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:02.008207 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:02.010216 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:02.010219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:02.010221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:02.010243 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:02.011525 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:02.011835 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:02.011838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:02.011839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:02.011901 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:02.012072 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:02.012074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:02.012075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:02.012286 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:02.012379 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:02.012543 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 07:07:02.012627 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 07:07:02.013099 11347 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47738
I20260620 07:07:02.013816 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11409
I20260620 07:07:02.018443 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11543
I20260620 07:07:02.022437 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11677
I20260620 07:07:02.026623 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11315
2026-06-20T07:07:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5045 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:02 dist-test-slave-wjw3 krb5kdc[11818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:02 dist-test-slave-wjw3 krb5kdc[11818](info): set up 2 sockets
Jun 20 07:07:02 dist-test-slave-wjw3 krb5kdc[11818](info): commencing operation
krb5kdc: starting...
W20260620 07:07:02.967180 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:03.071928 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37003
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:03.177767 11831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:03.178004 11831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:03.178066 11831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:03.181972 11831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:03.182021 11831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:03.182053 11831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:03.182075 11831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:03.182104 11831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:03.187425 11831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:37003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:03.188457 11831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:03.189404 11831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:03.197214 11836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.197263 11837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.197312 11831 server_base.cc:1034] running on GCE node
W20260620 07:07:03.197230 11839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.198036 11831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:03.199057 11831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:03.200201 11831 hybrid_clock.cc:648] HybridClock initialized: now 1781939223200188 us; error 26 us; skew 500 ppm
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:03.202696 11831 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:03.203504 11831 webserver.cc:460] Webserver started at http://127.8.24.62:35727/ using document root <none> and password file <none>
I20260620 07:07:03.203994 11831 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:03.204049 11831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:03.204208 11831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:03.206089 11831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "ba9495d8d57440ec8b2ef414bbd050f0"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "22dc39f1bb9b7eefcf02b449f80fef78"
server_key_iv: "a1896aaf24c6d13d4d01a15d5d239ed4"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.206595 11831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "ba9495d8d57440ec8b2ef414bbd050f0"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "22dc39f1bb9b7eefcf02b449f80fef78"
server_key_iv: "a1896aaf24c6d13d4d01a15d5d239ed4"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.209893 11831 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:07:03.212035 11846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.213213 11831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:03.213313 11831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ba9495d8d57440ec8b2ef414bbd050f0"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "22dc39f1bb9b7eefcf02b449f80fef78"
server_key_iv: "a1896aaf24c6d13d4d01a15d5d239ed4"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.213392 11831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:03.231264 11831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:03.234053 11831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:03.234203 11831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:03.240388 11831 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:37003
I20260620 07:07:03.240440 11898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:37003 every 8 connection(s)
I20260620 07:07:03.241232 11831 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:03.243825 11899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:03.246927 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11831
I20260620 07:07:03.247052 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:03.247314 8288 external_mini_cluster.cc:1428] Setting key 08f613db91b154c5e5289e63d225c552
I20260620 07:07:03.249344 11899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0: Bootstrap starting.
I20260620 07:07:03.251657 11899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:03.252432 11899 log.cc:824] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:03.254330 11899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0: No bootstrap required, opened a new log
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:03.257979 11899 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } }
I20260620 07:07:03.258239 11899 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:03.258294 11899 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9495d8d57440ec8b2ef414bbd050f0, State: Initialized, Role: FOLLOWER
I20260620 07:07:03.258683 11899 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } }
I20260620 07:07:03.258805 11899 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:03.258873 11899 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:03.258965 11899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:03.259853 11899 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } }
I20260620 07:07:03.260171 11899 leader_election.cc:302] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba9495d8d57440ec8b2ef414bbd050f0; no voters:
I20260620 07:07:03.260393 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.248573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.8.24.62:37003)
0620 07:07:03.248910 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.248924 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.248958 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:07:03.249558 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.249575 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.249878 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.250128 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.250951 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.251984 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.253050 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.253523 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.253637 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.256077 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.256082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.256093 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.256117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.257600 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.257968 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.257972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.257977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.258101 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.258360 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.258363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.258364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.258772 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.258932 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.259355 (+ 423us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.259640 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260620 07:07:03.260438 11899 leader_election.cc:288] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:03.260514 11904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:03.260701 11904 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [term 1 LEADER]: Becoming Leader. State: Replica: ba9495d8d57440ec8b2ef414bbd050f0, State: Running, Role: LEADER
I20260620 07:07:03.261050 11904 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } }
I20260620 07:07:03.261620 11899 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:03.262634 11905 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } } }
I20260620 07:07:03.262753 11905 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:03.263115 11913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:03.263091 11906 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba9495d8d57440ec8b2ef414bbd050f0. Latest consensus state: current_term: 1 leader_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9495d8d57440ec8b2ef414bbd050f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 37003 } } }
I20260620 07:07:03.263223 11906 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:03.266093 11913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:03.270731 11913 catalog_manager.cc:1347] Generated new cluster ID: 93bf79d1d1b24654a23e4ad4af81b052
I20260620 07:07:03.270787 11913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:03.281355 11913 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:03.282512 11913 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:03.289248 11913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba9495d8d57440ec8b2ef414bbd050f0: Generated new TSK 0
I20260620 07:07:03.289960 11913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:03.319397 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:03.423273 11925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:03.423556 11925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:03.423626 11925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:03.427546 11925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:03.427596 11925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:03.427670 11925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:03.432714 11925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:03.433676 11925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:03.434640 11925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:03.443540 11933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.443540 11930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.443549 11931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.443689 11925 server_base.cc:1034] running on GCE node
I20260620 07:07:03.444201 11925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:03.444782 11925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:03.445930 11925 hybrid_clock.cc:648] HybridClock initialized: now 1781939223445908 us; error 33 us; skew 500 ppm
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:03.448429 11925 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:03.449227 11925 webserver.cc:460] Webserver started at http://127.8.24.1:33397/ using document root <none> and password file <none>
I20260620 07:07:03.449746 11925 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:03.449810 11925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:03.449988 11925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:03.452020 11925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "6f6c4ca6c94147b4a9e03996416545cc"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "7b7c9537186c7b71e33a2982d4b9f628"
server_key_iv: "522ff648bb0e01954035dc56bd31cc5c"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.452545 11925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "6f6c4ca6c94147b4a9e03996416545cc"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "7b7c9537186c7b71e33a2982d4b9f628"
server_key_iv: "522ff648bb0e01954035dc56bd31cc5c"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.456070 11925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:03.458209 11940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.459290 11925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:03.459388 11925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "6f6c4ca6c94147b4a9e03996416545cc"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "7b7c9537186c7b71e33a2982d4b9f628"
server_key_iv: "522ff648bb0e01954035dc56bd31cc5c"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.459499 11925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:03.484351 11925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:03.487401 11925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:03.487614 11925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:03.488180 11925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:03.488965 11925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:03.489012 11925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.489063 11925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:03.489090 11925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.497908 11925 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:41105
I20260620 07:07:03.497951 12053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:41105 every 8 connection(s)
I20260620 07:07:03.498778 11925 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:03.504971 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 11925
I20260620 07:07:03.505112 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:03.505391 8288 external_mini_cluster.cc:1428] Setting key 5156bf1d3246515bc91003a8fe93dc02
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:03.511714 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.500485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:41921 (local address 127.8.24.62:37003)
0620 07:07:03.500605 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.500609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.501411 (+ 802us) server_negotiation.cc:407] Connection header received
0620 07:07:03.502687 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.502689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.502733 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.502802 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.503920 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.504441 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.505883 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.506123 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.506211 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.508604 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.508607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.508609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.508631 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.509860 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.510312 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.510315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.510316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.510386 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.510676 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.510679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.510681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.510846 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.510928 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.511393 (+ 465us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.511550 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":45}
I20260620 07:07:03.512480 12056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.500853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37003 (local address 127.8.24.1:41921)
0620 07:07:03.501272 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:03.501315 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:03.502497 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:03.502950 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:03.502956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:03.503291 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:03.503814 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.503828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.504530 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.504533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.505768 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.505778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.506255 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.506259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.506424 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.506942 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:03.506959 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:03.508445 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:03.509960 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.509966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.509981 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.510223 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.510462 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.510465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.510467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.510594 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.510937 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:03.510943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:03.511179 (+ 236us) client_negotiation.cc:769] Sending connection context
0620 07:07:03.511338 (+ 159us) client_negotiation.cc:240] Negotiation successful
0620 07:07:03.511668 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:07:03.513702 12054 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37003
I20260620 07:07:03.513908 12054 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:03.514371 12054 heartbeater.cc:511] Master 127.8.24.62:37003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:03.515802 11863 ts_manager.cc:194] Registered new tserver with Master: 6f6c4ca6c94147b4a9e03996416545cc (127.8.24.1:41105)
I20260620 07:07:03.516980 11863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:41921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:03.527482 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:03.631748 12059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:03.631978 12059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:03.632023 12059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:03.635962 12059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:03.636016 12059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:03.636086 12059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:03.641166 12059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:03.642146 12059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:03.643083 12059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:03.652097 12067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.652135 12064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.652098 12065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.652369 12059 server_base.cc:1034] running on GCE node
I20260620 07:07:03.652732 12059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:03.653317 12059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:03.654464 12059 hybrid_clock.cc:648] HybridClock initialized: now 1781939223654445 us; error 34 us; skew 500 ppm
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:03.656962 12059 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:03.657857 12059 webserver.cc:460] Webserver started at http://127.8.24.2:36823/ using document root <none> and password file <none>
I20260620 07:07:03.658370 12059 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:03.658432 12059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:03.658607 12059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:03.660570 12059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "867e17292c494cc2800a785a6e972b46"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "a533e4b1d3d4540bc5cb79ba05e7f664"
server_key_iv: "274391d5c6b1007f2f97adb652cb9dbd"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.661087 12059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "867e17292c494cc2800a785a6e972b46"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "a533e4b1d3d4540bc5cb79ba05e7f664"
server_key_iv: "274391d5c6b1007f2f97adb652cb9dbd"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.664723 12059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:03.666873 12074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.667999 12059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:03.668097 12059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "867e17292c494cc2800a785a6e972b46"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "a533e4b1d3d4540bc5cb79ba05e7f664"
server_key_iv: "274391d5c6b1007f2f97adb652cb9dbd"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.668179 12059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:03.678062 12059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:03.680554 12059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:03.680707 12059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:03.681209 12059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:03.681936 12059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:03.681980 12059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.682029 12059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:03.682055 12059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.690212 12059 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38667
I20260620 07:07:03.690243 12187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38667 every 8 connection(s)
I20260620 07:07:03.691030 12059 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:03.692577 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12059
I20260620 07:07:03.692682 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:03.692970 8288 external_mini_cluster.cc:1428] Setting key 8f19ce9bf9fe7e21efe153902fcddc4e
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:07:03.705186 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.692728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:33417 (local address 127.8.24.62:37003)
0620 07:07:03.692866 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.692871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.693676 (+ 805us) server_negotiation.cc:407] Connection header received
0620 07:07:03.695291 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.695294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.695334 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.695389 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.696726 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.697403 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.698967 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.699184 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.699257 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.701775 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.701777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.701779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.701801 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.703120 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.703618 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.703622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.703624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.703767 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.704118 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.704122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.704124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.704301 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.704384 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.704907 (+ 523us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.705020 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 07:07:03.705863 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.693102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37003 (local address 127.8.24.2:33417)
0620 07:07:03.693556 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:03.693598 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:03.695069 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:03.695610 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:03.695618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:03.696061 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:03.696588 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.696601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.697519 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.697523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.698857 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.698868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.699323 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.699327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.699555 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.700109 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:03.700130 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:03.701631 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:03.703216 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.703222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.703235 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.703523 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.703844 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.703847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.703849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.703982 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.704417 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:03.704428 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:03.704669 (+ 241us) client_negotiation.cc:769] Sending connection context
0620 07:07:03.704817 (+ 148us) client_negotiation.cc:240] Negotiation successful
0620 07:07:03.705055 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260620 07:07:03.706925 12188 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37003
I20260620 07:07:03.707130 12188 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:03.707613 12188 heartbeater.cc:511] Master 127.8.24.62:37003 requested a full tablet report, sending...
I20260620 07:07:03.708573 11863 ts_manager.cc:194] Registered new tserver with Master: 867e17292c494cc2800a785a6e972b46 (127.8.24.2:38667)
I20260620 07:07:03.709079 11863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:33417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:03.718029 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:03.822956 12193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:03.823189 12193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:03.823244 12193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:03.827140 12193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:03.827195 12193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:03.827272 12193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:03.832257 12193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:03.833214 12193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:03.834142 12193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:03.842792 12199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:03.842801 12198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.842872 12193 server_base.cc:1034] running on GCE node
W20260620 07:07:03.842809 12201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:03.843408 12193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:03.843976 12193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:03.845117 12193 hybrid_clock.cc:648] HybridClock initialized: now 1781939223845102 us; error 28 us; skew 500 ppm
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:03.847601 12193 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:03.848343 12193 webserver.cc:460] Webserver started at http://127.8.24.3:36459/ using document root <none> and password file <none>
I20260620 07:07:03.848848 12193 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:03.848912 12193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:03.849100 12193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:03.850997 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "8ee1373afd944395bca1469d82ddf132"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "23f3a268a3aad4bcc321211e2a1f8b19"
server_key_iv: "b68456b4e8e7114c27fe966b8f630702"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.851534 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "8ee1373afd944395bca1469d82ddf132"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "23f3a268a3aad4bcc321211e2a1f8b19"
server_key_iv: "b68456b4e8e7114c27fe966b8f630702"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.854897 12193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:07:03.857071 12208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.858146 12193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260620 07:07:03.858242 12193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "8ee1373afd944395bca1469d82ddf132"
format_stamp: "Formatted at 2026-06-20 07:07:03 on dist-test-slave-wjw3"
server_key: "23f3a268a3aad4bcc321211e2a1f8b19"
server_key_iv: "b68456b4e8e7114c27fe966b8f630702"
server_key_version: "encryptionkey@0"
I20260620 07:07:03.858327 12193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:03.870308 12193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:03.872926 12193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:03.873078 12193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:03.873580 12193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:03.874300 12193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:03.874357 12193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.874408 12193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:03.874432 12193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:03.882174 12193 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40233
I20260620 07:07:03.882194 12321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40233 every 8 connection(s)
I20260620 07:07:03.882942 12193 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:03.892827 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12193
I20260620 07:07:03.892957 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:03.893246 8288 external_mini_cluster.cc:1428] Setting key 09d988428980fe96e90b0b340035a133
I20260620 07:07:03.895709 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.884600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35817 (local address 127.8.24.62:37003)
0620 07:07:03.884743 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.884746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.885549 (+ 803us) server_negotiation.cc:407] Connection header received
0620 07:07:03.886856 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.886859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.886915 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.886984 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.888141 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.888666 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.889762 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.889982 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.890071 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.892324 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.892328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.892331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.892359 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.893780 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.894306 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.894309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.894310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.894410 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.894740 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.894742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.894743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.894908 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.894982 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.895498 (+ 516us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.895597 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 07:07:03.896456 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.884904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:37003 (local address 127.8.24.3:35817)
0620 07:07:03.885381 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:03.885423 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:03.886647 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:03.887160 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:03.887167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:03.887549 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:03.888002 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.888014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.888779 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.888782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.889645 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:03.889654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.890117 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.890121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:03.890272 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.890792 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:03.890808 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:03.892164 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:03.893883 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.893888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.893901 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.894216 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.894495 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:03.894498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:03.894499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:03.894662 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:03.895000 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:03.895006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:03.895275 (+ 269us) client_negotiation.cc:769] Sending connection context
0620 07:07:03.895419 (+ 144us) client_negotiation.cc:240] Negotiation successful
0620 07:07:03.895690 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":108,"threads_started":1}
I20260620 07:07:03.897434 12322 heartbeater.cc:347] Connected to a master server at 127.8.24.62:37003
I20260620 07:07:03.897630 12322 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:03.898056 12322 heartbeater.cc:511] Master 127.8.24.62:37003 requested a full tablet report, sending...
I20260620 07:07:03.898893 11863 ts_manager.cc:194] Registered new tserver with Master: 8ee1373afd944395bca1469d82ddf132 (127.8.24.3:40233)
I20260620 07:07:03.899394 11863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:35817
I20260620 07:07:03.906505 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:07:03.928910 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.920830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.8.24.1:41105)
0620 07:07:03.921116 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.921126 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.921147 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:07:03.921192 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.921197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.921367 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.921473 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.922051 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.922778 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.923573 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.923931 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.924012 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.926086 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.926093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.926106 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.926128 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.927542 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.927878 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.927882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.927883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.927952 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.928123 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.928125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.928126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.928327 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.928472 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.928606 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.928726 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
Jun 20 07:07:03 dist-test-slave-wjw3 krb5kdc[11818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:03.938697 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:03.931467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.8.24.62:37003)
0620 07:07:03.931580 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:03.931585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:03.931618 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:07:03.931735 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:03.931738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:03.931777 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:03.931839 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:03.932359 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.932871 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.933584 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:03.933792 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:03.933873 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:03.935809 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:03.935812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:03.935814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:03.935835 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:03.936887 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.937320 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.937323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.937325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.937467 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:03.937753 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:03.937756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:03.937757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:03.938024 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:03.938147 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:03.938423 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 07:07:03.938541 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 07:07:03.939643 11863 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48804
I20260620 07:07:03.940222 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11925
I20260620 07:07:03.945669 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12059
I20260620 07:07:03.949963 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12193
I20260620 07:07:03.954010 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 11831
2026-06-20T07:07:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1927 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:04 dist-test-slave-wjw3 krb5kdc[12334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:04 dist-test-slave-wjw3 krb5kdc[12334](info): set up 2 sockets
Jun 20 07:07:04 dist-test-slave-wjw3 krb5kdc[12334](info): commencing operation
krb5kdc: starting...
W20260620 07:07:04.903234 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:04 dist-test-slave-wjw3 krb5kdc[12334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:05.008527 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46653
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:05.113865 12347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:05.114120 12347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:05.114178 12347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:05.118163 12347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:05.118220 12347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:05.118247 12347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:05.118268 12347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:05.118296 12347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:05.123530 12347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:05.124567 12347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:05.125553 12347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:05.133440 12352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.133435 12355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.133435 12353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:05.133750 12347 server_base.cc:1034] running on GCE node
I20260620 07:07:05.134277 12347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:05.135313 12347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:05.136507 12347 hybrid_clock.cc:648] HybridClock initialized: now 1781939225136472 us; error 47 us; skew 500 ppm
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:05.139123 12347 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:05.139810 12347 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:05.139871 12347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:05.140038 12347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:05.142021 12347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "868e0f854d7b4603a1bb618001f5f1c5"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "741f39f41115d3f1a41bb50f4558e83e"
server_key_iv: "7d606a8a363876d09021cc021c6ce287"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.142508 12347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "868e0f854d7b4603a1bb618001f5f1c5"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "741f39f41115d3f1a41bb50f4558e83e"
server_key_iv: "7d606a8a363876d09021cc021c6ce287"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.146028 12347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:07:05.148268 12361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.149451 12347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:05.149572 12347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "868e0f854d7b4603a1bb618001f5f1c5"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "741f39f41115d3f1a41bb50f4558e83e"
server_key_iv: "7d606a8a363876d09021cc021c6ce287"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.149668 12347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:05.184080 12347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:05.187143 12347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:05.187310 12347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:05.193598 12347 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:46653
I20260620 07:07:05.193827 12413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:46653 every 8 connection(s)
I20260620 07:07:05.194188 12347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:05.196825 12414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:05.202098 12414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: Bootstrap starting.
I20260620 07:07:05.203686 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12347
I20260620 07:07:05.203812 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:05.204077 8288 external_mini_cluster.cc:1428] Setting key 5e3513de3b3ff9db8e319f256f72c214
I20260620 07:07:05.204775 12414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:05.205649 12414 log.cc:824] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:05.207504 12414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: No bootstrap required, opened a new log
I20260620 07:07:05.210778 12414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } }
I20260620 07:07:05.210994 12414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:05.211056 12414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e0f854d7b4603a1bb618001f5f1c5, State: Initialized, Role: FOLLOWER
I20260620 07:07:05.211468 12414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } }
I20260620 07:07:05.211612 12414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:05.211679 12414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:05.211776 12414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:05.212710 12414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } }
I20260620 07:07:05.213011 12414 leader_election.cc:302] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 868e0f854d7b4603a1bb618001f5f1c5; no voters:
I20260620 07:07:05.213361 12414 leader_election.cc:288] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:05.213438 12419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:05.213855 12419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [term 1 LEADER]: Becoming Leader. State: Replica: 868e0f854d7b4603a1bb618001f5f1c5, State: Running, Role: LEADER
I20260620 07:07:05.214131 12419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } }
I20260620 07:07:05.214625 12414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:05.217016 12420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "868e0f854d7b4603a1bb618001f5f1c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } } }
I20260620 07:07:05.217121 12420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:05.216754 12421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 868e0f854d7b4603a1bb618001f5f1c5. Latest consensus state: current_term: 1 leader_uuid: "868e0f854d7b4603a1bb618001f5f1c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868e0f854d7b4603a1bb618001f5f1c5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46653 } } }
I20260620 07:07:05.217381 12421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:05.217887 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.205122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.8.24.62:46653)
0620 07:07:05.205560 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.205575 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.205608 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:07:05.206350 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.206375 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.206701 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.207000 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:05.207785 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.208728 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.209573 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.210024 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.210166 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.212731 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:05.212736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:05.212749 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:05.212773 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:05.214432 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.214854 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.214860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.214866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.214970 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.215247 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.215250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.215251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.215758 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:05.215926 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.216729 (+ 803us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.217015 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":128,"threads_started":1}
W20260620 07:07:05.219396 12433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:05.219489 12433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:05.219565 12435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:05.220319 12435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:05.225286 12435 catalog_manager.cc:1347] Generated new cluster ID: 513647b83f734a8b922e8ba2eaf474c1
I20260620 07:07:05.225343 12435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:05.233253 12435 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:05.234045 12435 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:05.242761 12435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: Generated new TSK 0
I20260620 07:07:05.243539 12435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:05.267846 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:05.373066 12440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:05.373310 12440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:05.373353 12440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:05.377317 12440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:05.377372 12440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:05.377429 12440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:05.382480 12440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:05.383390 12440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:05.384335 12440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:05.393136 12446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.393174 12448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.393159 12445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:05.393357 12440 server_base.cc:1034] running on GCE node
I20260620 07:07:05.393712 12440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:05.394272 12440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:05.395416 12440 hybrid_clock.cc:648] HybridClock initialized: now 1781939225395390 us; error 38 us; skew 500 ppm
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:05.397967 12440 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:05.398619 12440 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:05.398679 12440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:05.398849 12440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:05.400743 12440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "478272269840495d98f69d108ab41bd0"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "29c9d67a9a09de6d046c9f0601923daa"
server_key_iv: "20511deb0dd03fe3877b930ec9c4f16c"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.401221 12440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "478272269840495d98f69d108ab41bd0"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "29c9d67a9a09de6d046c9f0601923daa"
server_key_iv: "20511deb0dd03fe3877b930ec9c4f16c"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.404639 12440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260620 07:07:05.406735 12454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.407917 12440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:05.408021 12440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "478272269840495d98f69d108ab41bd0"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "29c9d67a9a09de6d046c9f0601923daa"
server_key_iv: "20511deb0dd03fe3877b930ec9c4f16c"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.408113 12440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:05.426084 12440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:05.429175 12440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:05.429337 12440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:05.429895 12440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:05.430685 12440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:05.430729 12440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.430773 12440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:05.430800 12440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.438805 12440 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43615
I20260620 07:07:05.438808 12567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43615 every 8 connection(s)
I20260620 07:07:05.439484 12440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:05.443238 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12440
I20260620 07:07:05.443348 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:05.443648 8288 external_mini_cluster.cc:1428] Setting key 03e3fc50b023f4472e46b52c2bb81780
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:07:05.452374 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.441026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:52347 (local address 127.8.24.62:46653)
0620 07:07:05.441148 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.441151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.441978 (+ 827us) server_negotiation.cc:407] Connection header received
0620 07:07:05.443311 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.443314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.443371 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.443469 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:05.444611 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.445167 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.446354 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.446707 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.446809 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.449095 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:05.449099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:05.449101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:05.449124 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:05.450388 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.450928 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.450930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.450931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.450998 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.451329 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.451331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.451332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.451528 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:05.451601 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.452144 (+ 543us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.452255 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:07:05.453217 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.441395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46653 (local address 127.8.24.1:52347)
0620 07:07:05.441836 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.441877 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.443107 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.443640 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.443647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.444001 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:05.444468 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.444481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.445279 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.445282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.446196 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.446207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.446857 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.446862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.447041 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.447609 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:05.447625 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:05.448944 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:05.450493 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.450499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.450515 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.450843 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.451074 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.451078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.451080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.451247 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.451623 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:05.451629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:05.451920 (+ 291us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.452118 (+ 198us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.452468 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:05.454355 12568 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46653
I20260620 07:07:05.454540 12568 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:05.454909 12568 heartbeater.cc:511] Master 127.8.24.62:46653 requested a full tablet report, sending...
I20260620 07:07:05.456295 12375 ts_manager.cc:194] Registered new tserver with Master: 478272269840495d98f69d108ab41bd0 (127.8.24.1:43615)
I20260620 07:07:05.457350 12375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:52347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:05.467217 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:05.572790 12573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:05.573043 12573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:05.573086 12573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:05.577069 12573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:05.577121 12573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:05.577178 12573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:05.582145 12573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:05.583065 12573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:05.584043 12573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:05.592813 12581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.592800 12579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.592806 12578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:05.593003 12573 server_base.cc:1034] running on GCE node
I20260620 07:07:05.593451 12573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:05.594034 12573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:05.595180 12573 hybrid_clock.cc:648] HybridClock initialized: now 1781939225595160 us; error 35 us; skew 500 ppm
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:05.597765 12573 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:05.598448 12573 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:05.598515 12573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:05.598692 12573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:05.600660 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "a6b746817664424a971715a9d120a9a2"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "315d6e4e965c628df266ec7a7e9eeefa"
server_key_iv: "5252ea2487a12a1c2c5d3f9bd9f9aaff"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.601178 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "a6b746817664424a971715a9d120a9a2"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "315d6e4e965c628df266ec7a7e9eeefa"
server_key_iv: "5252ea2487a12a1c2c5d3f9bd9f9aaff"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.604678 12573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:05.606936 12587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.608111 12573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:05.608215 12573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "a6b746817664424a971715a9d120a9a2"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "315d6e4e965c628df266ec7a7e9eeefa"
server_key_iv: "5252ea2487a12a1c2c5d3f9bd9f9aaff"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.608306 12573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:05.629060 12573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:05.632061 12573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:05.632221 12573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:05.632748 12573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:05.633514 12573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:05.633562 12573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.633615 12573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:05.633639 12573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.641683 12573 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:43055
I20260620 07:07:05.641705 12700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:43055 every 8 connection(s)
I20260620 07:07:05.642231 12573 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:05.642684 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12573
I20260620 07:07:05.642802 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:05.643069 8288 external_mini_cluster.cc:1428] Setting key 1b774464bc7648a7d84cc65054b4c4d0
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:07:05.656212 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.643924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58701 (local address 127.8.24.62:46653)
0620 07:07:05.644041 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.644045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.644992 (+ 947us) server_negotiation.cc:407] Connection header received
0620 07:07:05.646370 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.646373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.646418 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.646474 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:05.647911 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.648530 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.650107 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.650309 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.650372 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.652917 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:05.652920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:05.652922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:05.652946 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:05.654135 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.654621 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.654624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.654625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.654699 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.655028 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.655031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.655033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.655277 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:05.655372 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.655918 (+ 546us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.656043 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:05.657191 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.644379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46653 (local address 127.8.24.2:58701)
0620 07:07:05.644868 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.644907 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.646140 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.646663 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.646672 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.647134 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:05.647779 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.647796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.648651 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.648654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.650005 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.650016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.650435 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.650439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.650637 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.651173 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:05.651192 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:05.652770 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:05.654238 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.654243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.654254 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.654534 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.654772 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.654775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.654777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.654946 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.655365 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:05.655369 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:05.655672 (+ 303us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.655854 (+ 182us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.656172 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":150,"threads_started":1}
I20260620 07:07:05.658309 12701 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46653
I20260620 07:07:05.658528 12701 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:05.658939 12701 heartbeater.cc:511] Master 127.8.24.62:46653 requested a full tablet report, sending...
I20260620 07:07:05.659929 12375 ts_manager.cc:194] Registered new tserver with Master: a6b746817664424a971715a9d120a9a2 (127.8.24.2:43055)
I20260620 07:07:05.660472 12375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:58701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:05.668038 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:40121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:05.772111 12706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:05.772360 12706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:05.772403 12706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:05.776300 12706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:05.776350 12706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:05.776410 12706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:05.781848 12706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:05.782780 12706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:05.783722 12706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:05.792429 12712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.792474 12711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:05.792429 12714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:05.792713 12706 server_base.cc:1034] running on GCE node
I20260620 07:07:05.793162 12706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:05.793857 12706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:05.795013 12706 hybrid_clock.cc:648] HybridClock initialized: now 1781939225795002 us; error 36 us; skew 500 ppm
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:05.797565 12706 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:05.798226 12706 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:05.798290 12706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:05.798468 12706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:05.800439 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "51ef6b738d6947929d4b28476075a507"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "320c083488118e37ae5748414dd2365d"
server_key_iv: "ccd406b7a8a0ecc3d256252b65a05e40"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.800946 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "51ef6b738d6947929d4b28476075a507"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "320c083488118e37ae5748414dd2365d"
server_key_iv: "ccd406b7a8a0ecc3d256252b65a05e40"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.804348 12706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 07:07:05.806545 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.807876 12706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:05.807996 12706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "51ef6b738d6947929d4b28476075a507"
format_stamp: "Formatted at 2026-06-20 07:07:05 on dist-test-slave-wjw3"
server_key: "320c083488118e37ae5748414dd2365d"
server_key_iv: "ccd406b7a8a0ecc3d256252b65a05e40"
server_key_version: "encryptionkey@0"
I20260620 07:07:05.808104 12706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:05.831817 12706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:05.834761 12706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:05.834908 12706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:05.835450 12706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:05.836261 12706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:05.836305 12706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.836344 12706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:05.836364 12706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:05.844431 12706 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37827
I20260620 07:07:05.844455 12833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37827 every 8 connection(s)
I20260620 07:07:05.844985 12706 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:05.853036 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12706
I20260620 07:07:05.853170 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:05.853453 8288 external_mini_cluster.cc:1428] Setting key 1826221ea23ba41d847d626b67f81c77
Jun 20 07:07:05 dist-test-slave-wjw3 krb5kdc[12334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:05.857659 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.846596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:42643 (local address 127.8.24.62:46653)
0620 07:07:05.846715 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.846718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.847540 (+ 822us) server_negotiation.cc:407] Connection header received
0620 07:07:05.848837 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.848841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.848895 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.848973 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:05.850091 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.850613 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.851752 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.851975 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.852067 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.854457 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:05.854460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:05.854463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:05.854491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:05.855884 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.856315 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.856319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.856321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.856392 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.856687 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.856690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.856692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.856864 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:05.856945 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.857397 (+ 452us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.857506 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:05.858410 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.846936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46653 (local address 127.8.24.3:42643)
0620 07:07:05.847375 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.847415 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.848635 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.849145 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.849152 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.849475 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:05.849949 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.849963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.850724 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.850726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.851646 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.851656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.852104 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.852108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.852266 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.852757 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:05.852777 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:05.854302 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:05.855981 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.855987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.856001 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.856230 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.856466 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:05.856469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:05.856471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:05.856609 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:05.856952 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:05.856958 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:05.857186 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.857340 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.857646 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":108,"threads_started":1}
I20260620 07:07:05.859383 12834 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46653
I20260620 07:07:05.859654 12834 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:05.860152 12834 heartbeater.cc:511] Master 127.8.24.62:46653 requested a full tablet report, sending...
I20260620 07:07:05.861269 12375 ts_manager.cc:194] Registered new tserver with Master: 51ef6b738d6947929d4b28476075a507 (127.8.24.3:37827)
I20260620 07:07:05.862015 12375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:42643
I20260620 07:07:05.866940 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:07:05.876669 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.868688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.8.24.62:46653)
0620 07:07:05.868808 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.868813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.869012 (+ 199us) server_negotiation.cc:407] Connection header received
0620 07:07:05.869136 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.869138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.869174 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.869232 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:05.869990 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.870476 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.871660 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.871981 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.872086 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.873436 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:05.873440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:05.873443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:05.873471 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:05.875019 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.875428 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.875452 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.875454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.875549 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:05.875778 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:05.875781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:05.875783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:05.876053 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:05.876181 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.876381 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.876492 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:07:05.879626 12375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:05.883791 12375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:05.907078 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.900317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.8.24.3:37827)
0620 07:07:05.900633 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.900648 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.901383 (+ 735us) server_negotiation.cc:407] Connection header received
0620 07:07:05.901480 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.901485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.901694 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.901848 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:05.902374 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.903615 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.905380 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906199 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906273 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906416 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.906809 (+ 393us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.906902 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
I20260620 07:07:05.907238 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.899682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43055 (local address 127.0.0.1:51128)
0620 07:07:05.900234 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.900280 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.900381 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.901862 (+ 1481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.901868 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.901893 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:05.902040 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.902046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.904185 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.904188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.905506 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.905517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906719 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.906851 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906884 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.906950 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.907098 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":63,"thread_start_us":86,"threads_started":1}
I20260620 07:07:05.907275 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.901189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.8.24.1:43615)
0620 07:07:05.901492 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.901501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.902257 (+ 756us) server_negotiation.cc:407] Connection header received
0620 07:07:05.902338 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.902343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.902547 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.902687 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:05.903053 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.904102 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.905642 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906430 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906538 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906656 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.907062 (+ 406us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.907148 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
I20260620 07:07:05.907653 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.899690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37827 (local address 127.0.0.1:39250)
0620 07:07:05.901274 (+ 1584us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.901288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.901384 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.902159 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.902161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.902175 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:05.902253 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.902258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.903768 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.903771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.905244 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.905254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906511 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.906626 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906659 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.907422 (+ 763us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.907521 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1519,"mutex_wait_us":24,"thread_start_us":111,"threads_started":1}
I20260620 07:07:05.907650 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.899767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.8.24.2:43055)
0620 07:07:05.900996 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.901007 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.901031 (+ 24us) server_negotiation.cc:407] Connection header received
0620 07:07:05.901480 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.901485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.901694 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.901848 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:05.902163 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.904039 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.905642 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906429 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906526 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906656 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.907397 (+ 741us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.907508 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":120,"threads_started":1}
I20260620 07:07:05.907955 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.901015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43615 (local address 127.0.0.1:38022)
0620 07:07:05.902076 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.902087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.902171 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.902833 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.902836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.902852 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:05.902943 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.902948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.904237 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.904240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.905513 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.905519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.906719 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.906723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.906863 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.906889 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.906950 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.907838 (+ 888us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":880,"thread_start_us":91,"threads_started":1}
I20260620 07:07:05.910331 12502 tablet_service.cc:1467] Processing CreateTablet for tablet 147c53deae3a445295e291544c847f12 (DEFAULT_TABLE table=test-table [id=a4a8b02fb4fd4d42a57609958f085cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:05.910359 12768 tablet_service.cc:1467] Processing CreateTablet for tablet 147c53deae3a445295e291544c847f12 (DEFAULT_TABLE table=test-table [id=a4a8b02fb4fd4d42a57609958f085cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:05.911327 12768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147c53deae3a445295e291544c847f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:05.911329 12502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147c53deae3a445295e291544c847f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:05.913115 12635 tablet_service.cc:1467] Processing CreateTablet for tablet 147c53deae3a445295e291544c847f12 (DEFAULT_TABLE table=test-table [id=a4a8b02fb4fd4d42a57609958f085cd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:05.913806 12635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 147c53deae3a445295e291544c847f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:05.918606 12851 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Bootstrap starting.
I20260620 07:07:05.919962 12852 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Bootstrap starting.
I20260620 07:07:05.921497 12851 tablet_bootstrap.cc:654] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:05.922075 12852 tablet_bootstrap.cc:654] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:05.922809 12851 log.cc:824] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:05.923264 12852 log.cc:824] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:05.925304 12852 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: No bootstrap required, opened a new log
I20260620 07:07:05.925503 12852 ts_tablet_manager.cc:1397] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 07:07:05.925534 12851 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: No bootstrap required, opened a new log
I20260620 07:07:05.925805 12851 ts_tablet_manager.cc:1397] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260620 07:07:05.926608 12855 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Bootstrap starting.
I20260620 07:07:05.928697 12855 tablet_bootstrap.cc:654] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:05.929152 12852 raft_consensus.cc:348] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.929443 12852 raft_consensus.cc:374] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:05.929499 12852 raft_consensus.cc:729] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b746817664424a971715a9d120a9a2, State: Initialized, Role: FOLLOWER
I20260620 07:07:05.929852 12855 log.cc:824] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:05.929978 12852 consensus_queue.cc:260] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.930635 12701 heartbeater.cc:503] Master 127.8.24.62:46653 was elected leader, sending a full tablet report...
I20260620 07:07:05.931021 12852 ts_tablet_manager.cc:1428] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:07:05.931676 12851 raft_consensus.cc:348] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.931970 12851 raft_consensus.cc:374] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:05.932052 12851 raft_consensus.cc:729] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478272269840495d98f69d108ab41bd0, State: Initialized, Role: FOLLOWER
I20260620 07:07:05.932476 12851 consensus_queue.cc:260] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.933247 12568 heartbeater.cc:503] Master 127.8.24.62:46653 was elected leader, sending a full tablet report...
I20260620 07:07:05.933441 12851 ts_tablet_manager.cc:1428] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260620 07:07:05.935741 12855 tablet_bootstrap.cc:492] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: No bootstrap required, opened a new log
I20260620 07:07:05.935941 12855 ts_tablet_manager.cc:1397] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260620 07:07:05.939540 12855 raft_consensus.cc:348] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.939803 12855 raft_consensus.cc:374] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:05.939862 12855 raft_consensus.cc:729] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ef6b738d6947929d4b28476075a507, State: Initialized, Role: FOLLOWER
I20260620 07:07:05.940232 12855 consensus_queue.cc:260] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
W20260620 07:07:05.940441 12569 tablet.cc:2369] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:05.941239 12855 ts_tablet_manager.cc:1428] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:07:05.941258 12834 heartbeater.cc:503] Master 127.8.24.62:46653 was elected leader, sending a full tablet report...
I20260620 07:07:05.967546 12859 raft_consensus.cc:482] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:05.967811 12859 raft_consensus.cc:504] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.969099 12859 leader_election.cc:288] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6b746817664424a971715a9d120a9a2 (127.8.24.2:43055), 478272269840495d98f69d108ab41bd0 (127.8.24.1:43615)
I20260620 07:07:05.973796 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.970235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60033 (local address 127.8.24.1:43615)
0620 07:07:05.970361 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.970366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.970378 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:05.970424 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.970426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.970471 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.970542 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:05.970815 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.971375 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.972532 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.973143 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.973236 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.973296 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.973604 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.973669 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:07:05.973796 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.969559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35487 (local address 127.8.24.2:43055)
0620 07:07:05.969838 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.969843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.969859 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:07:05.969915 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.969918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.969972 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.970056 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:05.970320 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.970982 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.972532 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.973139 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.973236 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.973296 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:05.973604 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 07:07:05.973670 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260620 07:07:05.973850 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.969278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43055 (local address 127.8.24.3:35487)
0620 07:07:05.969379 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.969394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.969491 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.970064 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.970067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.970080 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:05.970197 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.970203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.971073 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.971077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.972429 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.972436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.973256 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.973260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.973456 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.973516 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.973585 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.973693 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":157}
I20260620 07:07:05.973867 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.969428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43615 (local address 127.8.24.3:60033)
0620 07:07:05.969725 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:05.969737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:05.969809 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:05.970562 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:05.970566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:05.970587 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:05.970709 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.970715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.971493 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.971497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.972425 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:05.972442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:05.973256 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:05.973260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:05.973454 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:05.973513 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 07:07:05.973585 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:07:05.973693 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":94,"thread_start_us":110,"threads_started":1}
I20260620 07:07:05.974489 12655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147c53deae3a445295e291544c847f12" candidate_uuid: "51ef6b738d6947929d4b28476075a507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b746817664424a971715a9d120a9a2" is_pre_election: true
I20260620 07:07:05.974519 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147c53deae3a445295e291544c847f12" candidate_uuid: "51ef6b738d6947929d4b28476075a507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478272269840495d98f69d108ab41bd0" is_pre_election: true
I20260620 07:07:05.974763 12522 raft_consensus.cc:2454] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ef6b738d6947929d4b28476075a507 in term 0.
I20260620 07:07:05.974771 12655 raft_consensus.cc:2454] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ef6b738d6947929d4b28476075a507 in term 0.
I20260620 07:07:05.975185 12724 leader_election.cc:302] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478272269840495d98f69d108ab41bd0, 51ef6b738d6947929d4b28476075a507; no voters:
I20260620 07:07:05.975520 12859 raft_consensus.cc:2781] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:05.975589 12859 raft_consensus.cc:482] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:05.975627 12859 raft_consensus.cc:3037] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:05.976696 12859 raft_consensus.cc:504] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.976984 12859 leader_election.cc:288] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [CANDIDATE]: Term 1 election: Requested vote from peers a6b746817664424a971715a9d120a9a2 (127.8.24.2:43055), 478272269840495d98f69d108ab41bd0 (127.8.24.1:43615)
I20260620 07:07:05.977246 12655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147c53deae3a445295e291544c847f12" candidate_uuid: "51ef6b738d6947929d4b28476075a507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b746817664424a971715a9d120a9a2"
I20260620 07:07:05.977352 12655 raft_consensus.cc:3037] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:05.977491 12522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "147c53deae3a445295e291544c847f12" candidate_uuid: "51ef6b738d6947929d4b28476075a507" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478272269840495d98f69d108ab41bd0"
I20260620 07:07:05.977603 12522 raft_consensus.cc:3037] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:05.978598 12655 raft_consensus.cc:2454] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ef6b738d6947929d4b28476075a507 in term 1.
I20260620 07:07:05.978703 12522 raft_consensus.cc:2454] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ef6b738d6947929d4b28476075a507 in term 1.
I20260620 07:07:05.978880 12721 leader_election.cc:302] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ef6b738d6947929d4b28476075a507, a6b746817664424a971715a9d120a9a2; no voters:
I20260620 07:07:05.979055 12859 raft_consensus.cc:2781] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:05.979255 12859 raft_consensus.cc:686] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 1 LEADER]: Becoming Leader. State: Replica: 51ef6b738d6947929d4b28476075a507, State: Running, Role: LEADER
I20260620 07:07:05.979583 12859 consensus_queue.cc:237] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } }
I20260620 07:07:05.982336 12375 catalog_manager.cc:5495] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51ef6b738d6947929d4b28476075a507 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "51ef6b738d6947929d4b28476075a507" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51ef6b738d6947929d4b28476075a507" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:06.002054 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:05.999038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.8.24.3:37827)
0620 07:07:05.999134 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:05.999137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:05.999148 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:05.999274 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:05.999276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:05.999326 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:05.999409 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:05.999630 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:06.000128 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:06.000940 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:06.001133 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:06.001196 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:06.001402 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:06.001780 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:06.001862 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.001891 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.001943 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 07:07:06.007589 12655 raft_consensus.cc:1261] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 1 FOLLOWER]: Refusing update from remote peer 51ef6b738d6947929d4b28476075a507: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:06.007588 12522 raft_consensus.cc:1261] T 147c53deae3a445295e291544c847f12 P 478272269840495d98f69d108ab41bd0 [term 1 FOLLOWER]: Refusing update from remote peer 51ef6b738d6947929d4b28476075a507: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:06.008162 12859 consensus_queue.cc:1035] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6b746817664424a971715a9d120a9a2" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:06.008332 12861 consensus_queue.cc:1035] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478272269840495d98f69d108ab41bd0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:06.015532 12864 mvcc.cc:205] Tried to move back new op lower bound from 7298823069720301568 to 7298823069614764032. Current Snapshot: MvccSnapshot[applied={T|T < 7298823069720301568}]
I20260620 07:07:06.016150 12866 mvcc.cc:205] Tried to move back new op lower bound from 7298823069720301568 to 7298823069614764032. Current Snapshot: MvccSnapshot[applied={T|T < 7298823069720301568}]
I20260620 07:07:06.023974 12375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:06.024142 12375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:06.026302 12375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 868e0f854d7b4603a1bb618001f5f1c5: Sending DeleteTablet for 3 replicas of tablet 147c53deae3a445295e291544c847f12
I20260620 07:07:06.026961 12768 tablet_service.cc:1514] Processing DeleteTablet for tablet 147c53deae3a445295e291544c847f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:06 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:39250
I20260620 07:07:06.026966 12635 tablet_service.cc:1514] Processing DeleteTablet for tablet 147c53deae3a445295e291544c847f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:06 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51128
I20260620 07:07:06.027407 12874 tablet_replica.cc:330] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: stopping tablet replica
I20260620 07:07:06.027494 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12440
I20260620 07:07:06.027572 12502 tablet_service.cc:1514] Processing DeleteTablet for tablet 147c53deae3a445295e291544c847f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:06 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:38022
I20260620 07:07:06.031625 12875 tablet_replica.cc:330] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: stopping tablet replica
I20260620 07:07:06.031955 12875 raft_consensus.cc:2229] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:06.032248 12875 raft_consensus.cc:2258] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:06.032677 12874 raft_consensus.cc:2229] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:06.032984 12874 raft_consensus.cc:2258] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:07:06.033232 12365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:43615 (error 108)
I20260620 07:07:06.033939 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12573
I20260620 07:07:06.033967 12875 ts_tablet_manager.cc:1905] T 147c53deae3a445295e291544c847f12 P a6b746817664424a971715a9d120a9a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:06.034210 12874 ts_tablet_manager.cc:1905] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:06.036994 12874 ts_tablet_manager.cc:1918] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:06.037091 12874 log.cc:1196] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/147c53deae3a445295e291544c847f12
I20260620 07:07:06.037403 12874 ts_tablet_manager.cc:1939] T 147c53deae3a445295e291544c847f12 P 51ef6b738d6947929d4b28476075a507: Deleting consensus metadata
I20260620 07:07:06.038513 12364 catalog_manager.cc:4841] TS 51ef6b738d6947929d4b28476075a507 (127.8.24.3:37827): tablet 147c53deae3a445295e291544c847f12 (table test-table [id=a4a8b02fb4fd4d42a57609958f085cd8]) successfully deleted
I20260620 07:07:06.039383 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12706
I20260620 07:07:06.040549 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.040249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43615 (local address 127.0.0.1:38038)
0620 07:07:06.040407 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.040468 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260620 07:07:06.040556 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.040307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:43055 (local address 127.0.0.1:51130)
0620 07:07:06.040446 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.040489 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:43055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260620 07:07:06.040731 12362 catalog_manager.cc:4584] TS a6b746817664424a971715a9d120a9a2 (127.8.24.2:43055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 147c53deae3a445295e291544c847f12: Network error: Client connection negotiation failed: client connection to 127.8.24.2:43055: connect: Connection refused (error 111)
I20260620 07:07:06.043987 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12347
2026-06-20T07:07:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2089 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T07:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:06Z Disabled control of system clock
I20260620 07:07:06.071136 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44169
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 07:07:06.174974 12880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:06.175254 12880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:06.175307 12880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:06.179287 12880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:06.179345 12880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:06.179366 12880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:06.179387 12880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:06.179406 12880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:06.184064 12880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:06.185060 12880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:06.186359 12880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:07:06.194532 12880 server_base.cc:1034] running on GCE node
W20260620 07:07:06.195605 12885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.195577 12886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.195577 12888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:06.196242 12880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:06.197281 12880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:06.198432 12880 hybrid_clock.cc:648] HybridClock initialized: now 1781939226198417 us; error 28 us; skew 500 ppm
I20260620 07:07:06.200192 12880 webserver.cc:460] Webserver started at http://127.8.24.62:40421/ using document root <none> and password file <none>
I20260620 07:07:06.200718 12880 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:06.200780 12880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:06.200956 12880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:06.202884 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "5a33c5e5df264d99b0948564b9e0fc85"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "98b400a171b7c112ff0d99b2c11bfd75"
server_key_iv: "740a3e1a8a50d85666a9b0f75b54f9fc"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.203400 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5a33c5e5df264d99b0948564b9e0fc85"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "98b400a171b7c112ff0d99b2c11bfd75"
server_key_iv: "740a3e1a8a50d85666a9b0f75b54f9fc"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.206875 12880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:07:06.209196 12894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.210314 12880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260620 07:07:06.210415 12880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "5a33c5e5df264d99b0948564b9e0fc85"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "98b400a171b7c112ff0d99b2c11bfd75"
server_key_iv: "740a3e1a8a50d85666a9b0f75b54f9fc"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.210500 12880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:06.214864 12880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:06.215493 12880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:06.215647 12880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:06.221931 12880 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44169
I20260620 07:07:06.221946 12946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44169 every 8 connection(s)
I20260620 07:07:06.222856 12880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:06.225431 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12880
I20260620 07:07:06.225518 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:06.225775 8288 external_mini_cluster.cc:1428] Setting key b29e2a8b5b9deb38d527b398eb31d75f
I20260620 07:07:06.226105 12947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:06.231973 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.227087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.8.24.62:44169)
0620 07:07:06.227806 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.227826 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.227867 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:07:06.228835 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.228858 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.229305 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.229689 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.230404 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.230410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.230424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.230435 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.230493 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.230513 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.230643 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.230711 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.231081 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":111,"threads_started":1}
I20260620 07:07:06.233211 12947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: Bootstrap starting.
I20260620 07:07:06.235476 12947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:06.236271 12947 log.cc:824] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:06.238013 12947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: No bootstrap required, opened a new log
I20260620 07:07:06.241489 12947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } }
I20260620 07:07:06.241696 12947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:06.241735 12947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a33c5e5df264d99b0948564b9e0fc85, State: Initialized, Role: FOLLOWER
I20260620 07:07:06.242079 12947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } }
I20260620 07:07:06.242193 12947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:06.242237 12947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:06.242302 12947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:06.262126 12947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } }
I20260620 07:07:06.262559 12947 leader_election.cc:302] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a33c5e5df264d99b0948564b9e0fc85; no voters:
I20260620 07:07:06.262897 12947 leader_election.cc:288] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:06.263039 12952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:06.263291 12952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [term 1 LEADER]: Becoming Leader. State: Replica: 5a33c5e5df264d99b0948564b9e0fc85, State: Running, Role: LEADER
I20260620 07:07:06.263644 12952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } }
I20260620 07:07:06.264392 12947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:06.264892 12953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a33c5e5df264d99b0948564b9e0fc85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } } }
I20260620 07:07:06.265015 12953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:06.265491 12957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:06.265579 12954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a33c5e5df264d99b0948564b9e0fc85. Latest consensus state: current_term: 1 leader_uuid: "5a33c5e5df264d99b0948564b9e0fc85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33c5e5df264d99b0948564b9e0fc85" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44169 } } }
I20260620 07:07:06.265650 12954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:06.268596 12957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:06.274325 12957 catalog_manager.cc:1347] Generated new cluster ID: 7c99b519e7564043b0c46d181d197ac9
I20260620 07:07:06.274397 12957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:06.290407 12957 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:06.290701 12957 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:06.302196 12957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: Generated new TSK 0
I20260620 07:07:06.302814 12957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:06.305639 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--builtin_ntp_servers=127.8.24.20:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 07:07:06.423182 12971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:06.423416 12971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:06.423481 12971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:06.427328 12971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:06.427374 12971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:06.427426 12971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:06.431991 12971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:06.432945 12971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:06.434188 12971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:06.443043 12979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.443069 12976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.443176 12977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:06.443207 12971 server_base.cc:1034] running on GCE node
I20260620 07:07:06.443696 12971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:06.444264 12971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:06.445417 12971 hybrid_clock.cc:648] HybridClock initialized: now 1781939226445403 us; error 26 us; skew 500 ppm
I20260620 07:07:06.447093 12971 webserver.cc:460] Webserver started at http://127.8.24.1:42731/ using document root <none> and password file <none>
I20260620 07:07:06.447657 12971 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:06.447724 12971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:06.447922 12971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:06.449878 12971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "b0cdc83a561c4a5280074ea4b1fcbb89"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "45b97c494447303ecc58ac4053c393a9"
server_key_iv: "5ed52f4c3eed0f93574f3d411d33f313"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.450397 12971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "b0cdc83a561c4a5280074ea4b1fcbb89"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "45b97c494447303ecc58ac4053c393a9"
server_key_iv: "5ed52f4c3eed0f93574f3d411d33f313"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.453861 12971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260620 07:07:06.456092 12985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.457228 12971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:07:06.457333 12971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "b0cdc83a561c4a5280074ea4b1fcbb89"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "45b97c494447303ecc58ac4053c393a9"
server_key_iv: "5ed52f4c3eed0f93574f3d411d33f313"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.457419 12971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:06.461674 12971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:06.462220 12971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:06.462365 12971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:06.462848 12971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:06.463686 12971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:06.463729 12971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.463776 12971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:06.463804 12971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.471575 12971 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40157
I20260620 07:07:06.471621 13098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40157 every 8 connection(s)
I20260620 07:07:06.472374 12971 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:06.477588 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.473938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:35333 (local address 127.8.24.62:44169)
0620 07:07:06.474058 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.474062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.475048 (+ 986us) server_negotiation.cc:407] Connection header received
0620 07:07:06.476247 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.476250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.476286 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.476353 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.477260 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.477263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.477265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.477276 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.477294 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.477302 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.477367 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.477452 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.477487 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:07:06.478356 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.474296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44169 (local address 127.8.24.1:35333)
0620 07:07:06.474908 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.474945 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.476082 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.476492 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.476499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.476588 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.477035 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.477036 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.477075 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.477139 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.477360 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.477364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.477367 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.477479 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.477665 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260620 07:07:06.479385 13099 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44169
I20260620 07:07:06.479611 13099 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:06.479965 13099 heartbeater.cc:511] Master 127.8.24.62:44169 requested a full tablet report, sending...
I20260620 07:07:06.481276 12911 ts_manager.cc:194] Registered new tserver with Master: b0cdc83a561c4a5280074ea4b1fcbb89 (127.8.24.1:40157)
I20260620 07:07:06.481784 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 12971
I20260620 07:07:06.481876 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:06.482132 8288 external_mini_cluster.cc:1428] Setting key 6f9356636e6d1a14e672866a79e9b983
I20260620 07:07:06.483883 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--builtin_ntp_servers=127.8.24.20:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 07:07:06.588811 13102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:06.589040 13102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:06.589077 13102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:06.592931 13102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:06.592972 13102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:06.593024 13102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:06.597571 13102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:06.598510 13102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:06.599810 13102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:06.608709 13108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.608749 13107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:06.608933 13102 server_base.cc:1034] running on GCE node
W20260620 07:07:06.609015 13110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:06.609390 13102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:06.609961 13102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:06.611105 13102 hybrid_clock.cc:648] HybridClock initialized: now 1781939226611087 us; error 31 us; skew 500 ppm
I20260620 07:07:06.612888 13102 webserver.cc:460] Webserver started at http://127.8.24.2:42995/ using document root <none> and password file <none>
I20260620 07:07:06.613401 13102 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:06.613463 13102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:06.613685 13102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:06.615705 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "283f929d340d46fda7c6a55665b1d6a8"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "60f3e0d16babbc909792de60c85d4e75"
server_key_iv: "07e10d5c9f039149e803b8efbce34e30"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.616206 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "283f929d340d46fda7c6a55665b1d6a8"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "60f3e0d16babbc909792de60c85d4e75"
server_key_iv: "07e10d5c9f039149e803b8efbce34e30"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.619524 13102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 07:07:06.621620 13116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.622714 13102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:06.622807 13102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "283f929d340d46fda7c6a55665b1d6a8"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "60f3e0d16babbc909792de60c85d4e75"
server_key_iv: "07e10d5c9f039149e803b8efbce34e30"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.622889 13102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:06.627000 13102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:06.627573 13102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:06.627712 13102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:06.628183 13102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:06.629019 13102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:06.629070 13102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.629118 13102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:06.629143 13102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.636845 13102 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39055
I20260620 07:07:06.636883 13229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39055 every 8 connection(s)
I20260620 07:07:06.637665 13102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:06.639123 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13102
I20260620 07:07:06.639226 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:06.639532 8288 external_mini_cluster.cc:1428] Setting key 4ad9cafb418196babdb8f44ae277645f
I20260620 07:07:06.641477 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--builtin_ntp_servers=127.8.24.20:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 07:07:06.643743 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.639276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:58717 (local address 127.8.24.62:44169)
0620 07:07:06.639408 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.639413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.640579 (+ 1166us) server_negotiation.cc:407] Connection header received
0620 07:07:06.642125 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.642127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.642161 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.642232 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.643293 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.643296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.643299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.643312 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.643335 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.643345 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.643458 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.643576 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.643617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:07:06.644526 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.639809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44169 (local address 127.8.24.2:58717)
0620 07:07:06.640458 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.640496 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.641884 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.642399 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.642407 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.642515 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.643030 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.643032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.643072 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.643135 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.643453 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.643460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.643465 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.643596 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.643771 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260620 07:07:06.645525 13230 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44169
I20260620 07:07:06.645721 13230 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:06.646054 13230 heartbeater.cc:511] Master 127.8.24.62:44169 requested a full tablet report, sending...
I20260620 07:07:06.646970 12911 ts_manager.cc:194] Registered new tserver with Master: 283f929d340d46fda7c6a55665b1d6a8 (127.8.24.2:39055)
W20260620 07:07:06.745448 13233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:06.745674 13233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:06.745709 13233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:06.749553 13233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:06.749599 13233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:06.749650 13233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:06.754204 13233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:06.755192 13233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:06.756546 13233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:06.765437 13241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.765442 13239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:06.765595 13238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:06.765712 13233 server_base.cc:1034] running on GCE node
I20260620 07:07:06.766099 13233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:06.766690 13233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:06.767823 13233 hybrid_clock.cc:648] HybridClock initialized: now 1781939226767808 us; error 31 us; skew 500 ppm
I20260620 07:07:06.769368 13233 webserver.cc:460] Webserver started at http://127.8.24.3:33809/ using document root <none> and password file <none>
I20260620 07:07:06.769869 13233 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:06.769923 13233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:06.770108 13233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:06.772038 13233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "4dc465e5dd904d528d6007efdfa6be27"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "da1f46d952498646776cef0dc9ef87da"
server_key_iv: "138ab325f1c3d400006ba756a513457d"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.772541 13233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "4dc465e5dd904d528d6007efdfa6be27"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "da1f46d952498646776cef0dc9ef87da"
server_key_iv: "138ab325f1c3d400006ba756a513457d"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.775884 13233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:06.778054 13247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.779114 13233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:06.779210 13233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "4dc465e5dd904d528d6007efdfa6be27"
format_stamp: "Formatted at 2026-06-20 07:07:06 on dist-test-slave-wjw3"
server_key: "da1f46d952498646776cef0dc9ef87da"
server_key_iv: "138ab325f1c3d400006ba756a513457d"
server_key_version: "encryptionkey@0"
I20260620 07:07:06.779291 13233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:06.783368 13233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:06.783905 13233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:06.784042 13233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:06.784511 13233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:06.785300 13233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:06.785342 13233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.785389 13233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:06.785414 13233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:06.793066 13233 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:39437
I20260620 07:07:06.793087 13360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:39437 every 8 connection(s)
I20260620 07:07:06.793817 13233 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:06.796931 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13233
I20260620 07:07:06.797046 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:06.797278 8288 external_mini_cluster.cc:1428] Setting key f0356cf37863ac6c5d46c527e3c5adf0
I20260620 07:07:06.799307 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.795452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:57617 (local address 127.8.24.62:44169)
0620 07:07:06.795579 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.795582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.796518 (+ 936us) server_negotiation.cc:407] Connection header received
0620 07:07:06.797793 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.797796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.797839 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.797919 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.798911 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.798915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.798917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.798929 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.798950 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.798957 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.799025 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.799144 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.799186 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:06.800077 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.795793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44169 (local address 127.8.24.3:57617)
0620 07:07:06.796388 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.796425 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.797592 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.798079 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.798085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.798167 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.798672 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.798673 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.798710 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.798773 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.799022 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.799029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.799035 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.799177 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.799408 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":95,"threads_started":1}
I20260620 07:07:06.801071 13361 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44169
I20260620 07:07:06.801241 13361 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:06.801573 13361 heartbeater.cc:511] Master 127.8.24.62:44169 requested a full tablet report, sending...
I20260620 07:07:06.802456 12911 ts_manager.cc:194] Registered new tserver with Master: 4dc465e5dd904d528d6007efdfa6be27 (127.8.24.3:39437)
I20260620 07:07:06.810410 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:07:06.813336 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.812139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.8.24.62:44169)
0620 07:07:06.812265 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.812269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.812351 (+ 82us) server_negotiation.cc:407] Connection header received
0620 07:07:06.812447 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.812449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.812478 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.812535 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.813037 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.813040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.813042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.813051 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.813068 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.813075 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.813156 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.813195 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.813225 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:06.815757 12911 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:06.818565 12911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:06.828316 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.825792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.8.24.1:40157)
0620 07:07:06.826309 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.826321 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.826344 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:07:06.826388 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.826392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.826599 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.826735 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.827640 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.827646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.827658 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.827671 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.827715 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.827746 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.827826 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.828064 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.828146 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":123,"threads_started":1}
I20260620 07:07:06.828652 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.825644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40157 (local address 127.0.0.1:45672)
0620 07:07:06.826096 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.826145 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.826259 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.826789 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.826795 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.826868 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.827378 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.827380 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.827399 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.827495 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.827893 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.827902 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.827907 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.828463 (+ 556us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.828537 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":70,"threads_started":1}
I20260620 07:07:06.829214 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.826401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:39437 (local address 127.0.0.1:50208)
0620 07:07:06.827517 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.827539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.827603 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.827972 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.827975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.827990 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.828314 (+ 324us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.828316 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.828323 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.828340 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.828992 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.828995 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.828999 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.829067 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.829100 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":83,"threads_started":1}
I20260620 07:07:06.829299 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.826542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.8.24.3:39437)
0620 07:07:06.827616 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.827626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.827650 (+ 24us) server_negotiation.cc:407] Connection header received
0620 07:07:06.827697 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.827701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.827854 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.827976 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.828805 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.828811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.828820 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.828831 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.828869 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.828902 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.829004 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.829091 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.829153 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":138,"threads_started":1}
I20260620 07:07:06.829497 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.826763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.8.24.2:39055)
0620 07:07:06.827040 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:06.827051 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:06.827336 (+ 285us) server_negotiation.cc:407] Connection header received
0620 07:07:06.827392 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:06.827397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:06.827596 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:06.827738 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:06.828665 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:06.828669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:06.828676 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:06.828683 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:06.828926 (+ 243us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:06.828956 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:06.829054 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:06.829271 (+ 217us) server_negotiation.cc:299] Negotiation successful
0620 07:07:06.829341 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
I20260620 07:07:06.829792 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:06.825761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39055 (local address 127.0.0.1:38444)
0620 07:07:06.827163 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:06.827176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:06.827244 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:06.827714 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:06.827717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:06.827730 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:06.828177 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:06.828179 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:06.828189 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:06.828212 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:06.829089 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:06.829091 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:06.829094 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:07:06.829152 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 07:07:06.829177 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"thread_start_us":75,"threads_started":1}
I20260620 07:07:06.831729 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 (DEFAULT_TABLE table=test-table [id=2a9709ed880b4572871547895c9b624d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:06.832010 13164 tablet_service.cc:1467] Processing CreateTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 (DEFAULT_TABLE table=test-table [id=2a9709ed880b4572871547895c9b624d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:06.832314 13295 tablet_service.cc:1467] Processing CreateTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 (DEFAULT_TABLE table=test-table [id=2a9709ed880b4572871547895c9b624d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:06.832818 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821a764a90ba4a9bb20968fc53222ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:06.832844 13164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821a764a90ba4a9bb20968fc53222ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:06.833158 13295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 821a764a90ba4a9bb20968fc53222ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:06.839259 13378 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Bootstrap starting.
I20260620 07:07:06.839785 13379 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Bootstrap starting.
I20260620 07:07:06.840171 13380 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Bootstrap starting.
I20260620 07:07:06.841773 13378 tablet_bootstrap.cc:654] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:06.842199 13380 tablet_bootstrap.cc:654] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:06.842628 13378 log.cc:824] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:06.843003 13380 log.cc:824] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:06.844566 13380 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: No bootstrap required, opened a new log
I20260620 07:07:06.844672 13378 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: No bootstrap required, opened a new log
I20260620 07:07:06.844741 13380 ts_tablet_manager.cc:1397] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:07:06.844843 13378 ts_tablet_manager.cc:1397] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 07:07:06.844942 13379 tablet_bootstrap.cc:654] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:06.845705 13379 log.cc:824] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:06.847613 13379 tablet_bootstrap.cc:492] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: No bootstrap required, opened a new log
I20260620 07:07:06.847779 13379 ts_tablet_manager.cc:1397] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260620 07:07:06.848330 13380 raft_consensus.cc:348] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.848544 13380 raft_consensus.cc:374] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:06.848594 13380 raft_consensus.cc:729] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc465e5dd904d528d6007efdfa6be27, State: Initialized, Role: FOLLOWER
I20260620 07:07:06.848992 13380 consensus_queue.cc:260] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.848922 13378 raft_consensus.cc:348] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.849174 13378 raft_consensus.cc:374] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:06.849215 13378 raft_consensus.cc:729] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283f929d340d46fda7c6a55665b1d6a8, State: Initialized, Role: FOLLOWER
I20260620 07:07:06.849668 13378 consensus_queue.cc:260] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.849785 13380 ts_tablet_manager.cc:1428] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:07:06.850103 13361 heartbeater.cc:503] Master 127.8.24.62:44169 was elected leader, sending a full tablet report...
I20260620 07:07:06.850281 13230 heartbeater.cc:503] Master 127.8.24.62:44169 was elected leader, sending a full tablet report...
I20260620 07:07:06.850589 13378 ts_tablet_manager.cc:1428] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260620 07:07:06.852908 13379 raft_consensus.cc:348] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.853159 13379 raft_consensus.cc:374] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:06.853219 13379 raft_consensus.cc:729] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cdc83a561c4a5280074ea4b1fcbb89, State: Initialized, Role: FOLLOWER
I20260620 07:07:06.853626 13379 consensus_queue.cc:260] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:06.854472 13379 ts_tablet_manager.cc:1428] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260620 07:07:06.854790 13099 heartbeater.cc:503] Master 127.8.24.62:44169 was elected leader, sending a full tablet report...
W20260620 07:07:06.888478 13231 tablet.cc:2369] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:07:06.973397 13100 tablet.cc:2369] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:07:07.044660 13362 tablet.cc:2369] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:07.046417 13385 raft_consensus.cc:482] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:07.046579 13385 raft_consensus.cc:504] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:07.047672 13385 leader_election.cc:288] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0cdc83a561c4a5280074ea4b1fcbb89 (127.8.24.1:40157), 4dc465e5dd904d528d6007efdfa6be27 (127.8.24.3:39437)
I20260620 07:07:07.052341 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.047875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40157 (local address 127.8.24.2:36427)
0620 07:07:07.047989 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:07.048005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:07.048102 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:07.048318 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:07.048320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:07.048332 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:07.051836 (+ 3504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:07.051837 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:07.051847 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:07.051870 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:07.052114 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:07.052116 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:07.052119 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:07:07.052189 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:07:07.052217 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 07:07:07.052341 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.048155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37221 (local address 127.8.24.3:39437)
0620 07:07:07.048250 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.048253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.048447 (+ 194us) server_negotiation.cc:407] Connection header received
0620 07:07:07.048496 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.048499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.048537 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.048595 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.051970 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.051972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.051974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.051985 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.052002 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.052009 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.052078 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.052172 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.052214 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 07:07:07.052353 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.047972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:36427 (local address 127.8.24.1:40157)
0620 07:07:07.048108 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.048113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.048130 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:07:07.048181 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.048184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.048223 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.048300 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.051993 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.051996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.051998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.052009 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.052028 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.052036 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.052120 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.052190 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.052227 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:07:07.052385 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.047963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:39437 (local address 127.8.24.2:37221)
0620 07:07:07.048355 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:07.048367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:07.048440 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:07.048624 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:07.048626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:07.048639 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:07.051814 (+ 3175us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:07.051817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:07.051829 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:07.051856 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:07.052084 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:07.052087 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:07.052091 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:07.052163 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:07:07.052199 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":115,"threads_started":1}
I20260620 07:07:07.053077 13053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821a764a90ba4a9bb20968fc53222ab9" candidate_uuid: "283f929d340d46fda7c6a55665b1d6a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" is_pre_election: true
I20260620 07:07:07.053078 13315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821a764a90ba4a9bb20968fc53222ab9" candidate_uuid: "283f929d340d46fda7c6a55665b1d6a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc465e5dd904d528d6007efdfa6be27" is_pre_election: true
I20260620 07:07:07.053395 13315 raft_consensus.cc:2454] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283f929d340d46fda7c6a55665b1d6a8 in term 0.
I20260620 07:07:07.053395 13053 raft_consensus.cc:2454] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283f929d340d46fda7c6a55665b1d6a8 in term 0.
I20260620 07:07:07.053862 13120 leader_election.cc:302] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283f929d340d46fda7c6a55665b1d6a8, 4dc465e5dd904d528d6007efdfa6be27; no voters:
I20260620 07:07:07.054139 13385 raft_consensus.cc:2781] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:07.054214 13385 raft_consensus.cc:482] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:07.054244 13385 raft_consensus.cc:3037] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.055248 13385 raft_consensus.cc:504] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:07.055615 13385 leader_election.cc:288] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [CANDIDATE]: Term 1 election: Requested vote from peers b0cdc83a561c4a5280074ea4b1fcbb89 (127.8.24.1:40157), 4dc465e5dd904d528d6007efdfa6be27 (127.8.24.3:39437)
I20260620 07:07:07.055929 13053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821a764a90ba4a9bb20968fc53222ab9" candidate_uuid: "283f929d340d46fda7c6a55665b1d6a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89"
I20260620 07:07:07.055933 13315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "821a764a90ba4a9bb20968fc53222ab9" candidate_uuid: "283f929d340d46fda7c6a55665b1d6a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc465e5dd904d528d6007efdfa6be27"
I20260620 07:07:07.056032 13315 raft_consensus.cc:3037] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.056041 13053 raft_consensus.cc:3037] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.056981 13053 raft_consensus.cc:2454] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283f929d340d46fda7c6a55665b1d6a8 in term 1.
I20260620 07:07:07.057030 13315 raft_consensus.cc:2454] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283f929d340d46fda7c6a55665b1d6a8 in term 1.
I20260620 07:07:07.057276 13120 leader_election.cc:302] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283f929d340d46fda7c6a55665b1d6a8, 4dc465e5dd904d528d6007efdfa6be27; no voters:
I20260620 07:07:07.057440 13385 raft_consensus.cc:2781] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:07.057642 13385 raft_consensus.cc:686] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 1 LEADER]: Becoming Leader. State: Replica: 283f929d340d46fda7c6a55665b1d6a8, State: Running, Role: LEADER
I20260620 07:07:07.057972 13385 consensus_queue.cc:237] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } }
I20260620 07:07:07.061051 12910 catalog_manager.cc:5495] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283f929d340d46fda7c6a55665b1d6a8 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "283f929d340d46fda7c6a55665b1d6a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "283f929d340d46fda7c6a55665b1d6a8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:07.123842 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.122580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.8.24.2:39055)
0620 07:07:07.122701 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.122704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.122717 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:07.122794 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.122796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.122856 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.122919 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.123502 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.123505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.123507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.123517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.123535 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.123543 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.123617 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.123675 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.123708 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:07:07.129220 13315 raft_consensus.cc:1261] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 1 FOLLOWER]: Refusing update from remote peer 283f929d340d46fda7c6a55665b1d6a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:07.129222 13053 raft_consensus.cc:1261] T 821a764a90ba4a9bb20968fc53222ab9 P b0cdc83a561c4a5280074ea4b1fcbb89 [term 1 FOLLOWER]: Refusing update from remote peer 283f929d340d46fda7c6a55665b1d6a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:07.129848 13388 consensus_queue.cc:1035] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc465e5dd904d528d6007efdfa6be27" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 39437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:07.130008 13385 consensus_queue.cc:1035] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0cdc83a561c4a5280074ea4b1fcbb89" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:07.143743 12910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:07.143914 12910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:07.146143 12910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a33c5e5df264d99b0948564b9e0fc85: Sending DeleteTablet for 3 replicas of tablet 821a764a90ba4a9bb20968fc53222ab9
I20260620 07:07:07.146804 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:07 UTC) from {username='slave'} at 127.0.0.1:45672
I20260620 07:07:07.146971 13164 tablet_service.cc:1514] Processing DeleteTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:07 UTC) from {username='slave'} at 127.0.0.1:38444
I20260620 07:07:07.147394 13295 tablet_service.cc:1514] Processing DeleteTablet for tablet 821a764a90ba4a9bb20968fc53222ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:07 UTC) from {username='slave'} at 127.0.0.1:50208
I20260620 07:07:07.147519 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12971
I20260620 07:07:07.148247 13402 tablet_replica.cc:330] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: stopping tablet replica
I20260620 07:07:07.148523 13402 raft_consensus.cc:2229] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:07.148835 13402 raft_consensus.cc:2258] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:07.149945 13402 ts_tablet_manager.cc:1905] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:07.152029 13402 ts_tablet_manager.cc:1918] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:07.152112 13402 log.cc:1196] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/821a764a90ba4a9bb20968fc53222ab9
I20260620 07:07:07.152343 13402 ts_tablet_manager.cc:1939] T 821a764a90ba4a9bb20968fc53222ab9 P 283f929d340d46fda7c6a55665b1d6a8: Deleting consensus metadata
I20260620 07:07:07.153199 12898 catalog_manager.cc:4841] TS 283f929d340d46fda7c6a55665b1d6a8 (127.8.24.2:39055): tablet 821a764a90ba4a9bb20968fc53222ab9 (table test-table [id=2a9709ed880b4572871547895c9b624d]) successfully deleted
W20260620 07:07:07.153234 12895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:40157 (error 108)
I20260620 07:07:07.154017 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13102
I20260620 07:07:07.154078 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.153831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40157 (local address 127.0.0.1:45688)
0620 07:07:07.153937 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:07.154000 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 07:07:07.154109 13403 tablet_replica.cc:330] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: stopping tablet replica
I20260620 07:07:07.154379 13403 raft_consensus.cc:2229] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:07.154637 13403 raft_consensus.cc:2258] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:07:07.155311 12895 catalog_manager.cc:4584] TS b0cdc83a561c4a5280074ea4b1fcbb89 (127.8.24.1:40157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 821a764a90ba4a9bb20968fc53222ab9: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40157: connect: Connection refused (error 111)
I20260620 07:07:07.155934 13403 ts_tablet_manager.cc:1905] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:07.157830 13403 ts_tablet_manager.cc:1918] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:07.157902 13403 log.cc:1196] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/821a764a90ba4a9bb20968fc53222ab9
I20260620 07:07:07.158145 13403 ts_tablet_manager.cc:1939] T 821a764a90ba4a9bb20968fc53222ab9 P 4dc465e5dd904d528d6007efdfa6be27: Deleting consensus metadata
I20260620 07:07:07.159049 12898 catalog_manager.cc:4841] TS 4dc465e5dd904d528d6007efdfa6be27 (127.8.24.3:39437): tablet 821a764a90ba4a9bb20968fc53222ab9 (table test-table [id=2a9709ed880b4572871547895c9b624d]) successfully deleted
I20260620 07:07:07.159339 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13233
I20260620 07:07:07.163610 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 12880
2026-06-20T07:07:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1118 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 07:07:07.182817 8288 mini_oidc.cc:152] Starting JWKS server
I20260620 07:07:07.184404 8288 webserver.cc:460] Webserver started at https://127.0.0.1:34095/ using document root <none> and password file <none>
I20260620 07:07:07.184563 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 07:07:07.184803 8288 webserver.cc:460] Webserver started at http://127.0.0.1:43939/ using document root <none> and password file <none>
2026-06-20T07:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:07Z Disabled control of system clock
I20260620 07:07:07.192829 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45485
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:39083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45485
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34095/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/testchainca.cert with env {}
W20260620 07:07:07.296149 13410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:07.296387 13410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:07.296423 13410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:07.300390 13410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:07.300437 13410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:07.300451 13410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:07.300467 13410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:07.300483 13410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 07:07:07.300503 13410 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 07:07:07.300518 13410 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34095/jwks.json
I20260620 07:07:07.305576 13410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34095/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:07.306474 13410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:07.307416 13410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:07.315187 13416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:07.315196 13418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:07.315198 13415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:07.315240 13410 server_base.cc:1034] running on GCE node
I20260620 07:07:07.315925 13410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:07.316957 13410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:07.318106 13410 hybrid_clock.cc:648] HybridClock initialized: now 1781939227318091 us; error 29 us; skew 500 ppm
I20260620 07:07:07.319754 13410 webserver.cc:460] Webserver started at http://127.8.24.62:38745/ using document root <none> and password file <none>
I20260620 07:07:07.320283 13410 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:07.320346 13410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:07.320528 13410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:07.322414 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "4b2e53df7a5f43fc91b28b3ecdacf996"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "e0d94ba4d02ba1042c88be9e2eb1af40"
server_key_iv: "baeffab95b4f4762f2c0e53fade78668"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.322937 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "4b2e53df7a5f43fc91b28b3ecdacf996"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "e0d94ba4d02ba1042c88be9e2eb1af40"
server_key_iv: "baeffab95b4f4762f2c0e53fade78668"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.326290 13410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:07:07.328510 13424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:07.329602 13410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:07.329706 13410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "4b2e53df7a5f43fc91b28b3ecdacf996"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "e0d94ba4d02ba1042c88be9e2eb1af40"
server_key_iv: "baeffab95b4f4762f2c0e53fade78668"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.329792 13410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:07.352687 13410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:07.353367 13410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:07.353538 13410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:07.360157 13410 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45485
I20260620 07:07:07.360174 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45485 every 8 connection(s)
I20260620 07:07:07.361044 13410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:07.363710 13477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:07.367743 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13410
I20260620 07:07:07.367835 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:07.368059 8288 external_mini_cluster.cc:1428] Setting key caf3618efa018b2e06a294b4049b856a
I20260620 07:07:07.369081 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996: Bootstrap starting.
I20260620 07:07:07.371136 13477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:07.371914 13477 log.cc:824] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:07.373827 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996: No bootstrap required, opened a new log
I20260620 07:07:07.376819 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.369328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.8.24.62:45485)
0620 07:07:07.369909 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.369922 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.369956 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:07:07.370544 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.370560 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.370931 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.371206 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.371610 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.372730 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.373848 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.374313 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.374438 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.375194 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.375199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.375210 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.375607 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.375659 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.375679 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.375749 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.375885 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.376135 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":100,"threads_started":1}
I20260620 07:07:07.377421 13477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } }
I20260620 07:07:07.377630 13477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:07.377686 13477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2e53df7a5f43fc91b28b3ecdacf996, State: Initialized, Role: FOLLOWER
I20260620 07:07:07.378073 13477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } }
I20260620 07:07:07.378185 13477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:07.378240 13477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:07.378331 13477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.379136 13477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } }
I20260620 07:07:07.379259 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:07:07.379467 13477 leader_election.cc:302] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b2e53df7a5f43fc91b28b3ecdacf996; no voters:
I20260620 07:07:07.379753 13477 leader_election.cc:288] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:07.379848 13482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:07.380048 13482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [term 1 LEADER]: Becoming Leader. State: Replica: 4b2e53df7a5f43fc91b28b3ecdacf996, State: Running, Role: LEADER
I20260620 07:07:07.380339 13482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } }
I20260620 07:07:07.380872 13477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:07.381906 13483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } } }
I20260620 07:07:07.382025 13483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:07.382296 13484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b2e53df7a5f43fc91b28b3ecdacf996. Latest consensus state: current_term: 1 leader_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2e53df7a5f43fc91b28b3ecdacf996" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45485 } } }
I20260620 07:07:07.382395 13484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:07.382491 13491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:07.385063 13491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:07.389698 13491 catalog_manager.cc:1347] Generated new cluster ID: 25e73082c27f4657b20868073463fbe2
I20260620 07:07:07.389757 13491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:07.410185 13491 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:07.411357 13491 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:07.419123 13491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b2e53df7a5f43fc91b28b3ecdacf996: Generated new TSK 0
I20260620 07:07:07.419787 13491 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 07:07:07.448839 13507 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 07:07:07.448956 13507 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 07:07:07.449437 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.432701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.8.24.62:45485)
0620 07:07:07.432814 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.432818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.433001 (+ 183us) server_negotiation.cc:407] Connection header received
0620 07:07:07.433238 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.433240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.433284 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.433362 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0620 07:07:07.433634 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.434234 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.435409 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.435684 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.435757 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.436064 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 07:07:07.449098 (+ 13034us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 07:07:07.449321 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33962: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 07:07:07.449512 13480 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33962: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 07:07:07.450870 13506 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:07.432635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45485 (local address 127.0.0.1:33962)
0620 07:07:07.432924 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:07.432938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:07.433094 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:07.433386 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:07.433389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:07.433396 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 07:07:07.433519 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:07.433526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.434326 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.434330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:07.435314 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:07.435324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.435761 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.435778 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:07.435871 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.435918 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 07:07:07.449311 (+ 13393us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 07:07:07.449551 (+ 240us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:45485: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":83,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.24.62:45485: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34095/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 07:07:07.451541 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13410
2026-06-20T07:07:07Z chronyd exiting
I20260620 07:07:07.487659 8288 test_util.cc:175] -----------------------------------------------
I20260620 07:07:07.487732 8288 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781939200335300-8288-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 07:07:07.488590 8288 mini_oidc.cc:152] Starting JWKS server
I20260620 07:07:07.489346 8288 webserver.cc:460] Webserver started at https://127.0.0.1:41809/ using document root <none> and password file <none>
I20260620 07:07:07.489423 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 07:07:07.489629 8288 webserver.cc:460] Webserver started at http://127.0.0.1:38885/ using document root <none> and password file <none>
2026-06-20T07:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:07Z Disabled control of system clock
I20260620 07:07:07.497484 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42995
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:40811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41809/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/testchainca.cert with env {}
W20260620 07:07:07.600437 13513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:07.600672 13513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:07.600706 13513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:07.604604 13513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:07.604653 13513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:07.604667 13513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:07.604688 13513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:07.604701 13513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 07:07:07.604714 13513 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 07:07:07.604732 13513 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41809/jwks.json
I20260620 07:07:07.610149 13513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41809/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:07.611040 13513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:07.612005 13513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:07.619521 13518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:07.619561 13519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:07.619532 13521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:07.619801 13513 server_base.cc:1034] running on GCE node
I20260620 07:07:07.620290 13513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:07.621294 13513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:07.622447 13513 hybrid_clock.cc:648] HybridClock initialized: now 1781939227622420 us; error 38 us; skew 500 ppm
I20260620 07:07:07.624087 13513 webserver.cc:460] Webserver started at http://127.8.24.62:39183/ using document root <none> and password file <none>
I20260620 07:07:07.624603 13513 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:07.624666 13513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:07.624835 13513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:07.626726 13513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "b562a34180a94393acb64a64c62923f0"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d8ac848c28eaabeb020d5b6a4bedff82"
server_key_iv: "1524eff9d203bd6d539398e124b5744f"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.627244 13513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "b562a34180a94393acb64a64c62923f0"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d8ac848c28eaabeb020d5b6a4bedff82"
server_key_iv: "1524eff9d203bd6d539398e124b5744f"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.630626 13513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:07:07.632871 13527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:07.633987 13513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:07.634095 13513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "b562a34180a94393acb64a64c62923f0"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d8ac848c28eaabeb020d5b6a4bedff82"
server_key_iv: "1524eff9d203bd6d539398e124b5744f"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.634186 13513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:07.645681 13513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:07.646314 13513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:07.646478 13513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:07.652664 13513 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:42995
I20260620 07:07:07.652675 13579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:42995 every 8 connection(s)
I20260620 07:07:07.653502 13513 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:07.656329 13580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:07.661586 13580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: Bootstrap starting.
I20260620 07:07:07.662180 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13513
I20260620 07:07:07.662271 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:07.662494 8288 external_mini_cluster.cc:1428] Setting key f286aea602c081c12827714061c7d5a8
I20260620 07:07:07.664187 13580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:07.664851 13580 log.cc:824] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:07.666504 13580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: No bootstrap required, opened a new log
I20260620 07:07:07.669812 13580 raft_consensus.cc:348] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } }
I20260620 07:07:07.670017 13580 raft_consensus.cc:374] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:07.670068 13580 raft_consensus.cc:729] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b562a34180a94393acb64a64c62923f0, State: Initialized, Role: FOLLOWER
I20260620 07:07:07.670476 13580 consensus_queue.cc:260] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } }
I20260620 07:07:07.670603 13580 raft_consensus.cc:388] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:07.670663 13580 raft_consensus.cc:482] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:07.670763 13580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.670991 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.663687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.8.24.62:42995)
0620 07:07:07.664266 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.664282 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.664324 (+ 42us) server_negotiation.cc:407] Connection header received
0620 07:07:07.664916 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.664931 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.665332 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.665632 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.666023 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.666981 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.667886 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.668343 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.668484 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.669322 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.669327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.669338 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.669703 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.669756 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.669775 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.669858 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.669987 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.670249 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260620 07:07:07.671723 13580 raft_consensus.cc:504] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } }
I20260620 07:07:07.672048 13580 leader_election.cc:302] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b562a34180a94393acb64a64c62923f0; no voters:
I20260620 07:07:07.672302 13580 leader_election.cc:288] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:07.672384 13585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:07.672588 13585 raft_consensus.cc:686] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [term 1 LEADER]: Becoming Leader. State: Replica: b562a34180a94393acb64a64c62923f0, State: Running, Role: LEADER
I20260620 07:07:07.672876 13585 consensus_queue.cc:237] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } }
I20260620 07:07:07.673444 13580 sys_catalog.cc:564] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:07.673975 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:07:07.675918 13586 sys_catalog.cc:455] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b562a34180a94393acb64a64c62923f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } } }
I20260620 07:07:07.676095 13587 sys_catalog.cc:455] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b562a34180a94393acb64a64c62923f0. Latest consensus state: current_term: 1 leader_uuid: "b562a34180a94393acb64a64c62923f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b562a34180a94393acb64a64c62923f0" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42995 } } }
I20260620 07:07:07.676239 13587 sys_catalog.cc:458] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:07.676257 13586 sys_catalog.cc:458] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0 [sys.catalog]: This master's current role is: LEADER
W20260620 07:07:07.677914 13600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:07.677976 13600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:07.678089 13601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:07.678915 13601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:07.683640 13601 catalog_manager.cc:1347] Generated new cluster ID: c4c0274400414feea9c47f4eb5eb6d62
I20260620 07:07:07.683697 13601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:07.707805 13601 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:07.708658 13601 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:07.716626 13601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b562a34180a94393acb64a64c62923f0: Generated new TSK 0
I20260620 07:07:07.717322 13601 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 07:07:07.756253 13610 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 07:07:07.756335 13610 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 07:07:07.756946 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.734013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.8.24.62:42995)
0620 07:07:07.734128 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.734133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.734308 (+ 175us) server_negotiation.cc:407] Connection header received
0620 07:07:07.734475 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.734477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.734521 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.734613 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0620 07:07:07.734842 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.735369 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.736225 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.736438 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.736520 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.736806 (+ 286us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 07:07:07.756552 (+ 19746us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41809/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 07:07:07.756805 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50670: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41809/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260620 07:07:07.757021 13583 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50670: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41809/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 07:07:07.757047 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:07.733950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42995 (local address 127.0.0.1:50670)
0620 07:07:07.734230 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:07.734241 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:07.734342 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:07.734625 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:07.734628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:07.734635 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 07:07:07.734747 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:07.734752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.735488 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.735492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:07.736119 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:07.736126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.736520 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.736548 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:07.736635 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.736657 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 07:07:07.756734 (+ 20077us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41809/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 07:07:07.756889 (+ 155us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:42995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41809/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":111,"threads_started":1}
I20260620 07:07:07.757905 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13513
2026-06-20T07:07:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (307 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 07:07:07.796259 8288 mini_oidc.cc:152] Starting JWKS server
I20260620 07:07:07.797042 8288 webserver.cc:460] Webserver started at https://127.0.0.1:40623/ using document root <none> and password file <none>
I20260620 07:07:07.797124 8288 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 07:07:07.797323 8288 webserver.cc:460] Webserver started at http://127.0.0.1:38789/ using document root <none> and password file <none>
2026-06-20T07:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:07Z Disabled control of system clock
I20260620 07:07:07.805132 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38337
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38337
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40623/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 07:07:07.907881 13616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:07.908114 13616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:07.908155 13616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:07.912016 13616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:07.912058 13616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:07.912072 13616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:07.912086 13616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:07.912103 13616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 07:07:07.912120 13616 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 07:07:07.912134 13616 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40623/jwks.json
I20260620 07:07:07.916606 13616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40623/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:07.917486 13616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:07.918705 13616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:07.926151 13622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:07.926177 13621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:07.926263 13616 server_base.cc:1034] running on GCE node
W20260620 07:07:07.926146 13624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:07.926898 13616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:07.927925 13616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:07.929071 13616 hybrid_clock.cc:648] HybridClock initialized: now 1781939227929052 us; error 34 us; skew 500 ppm
I20260620 07:07:07.930685 13616 webserver.cc:460] Webserver started at http://127.8.24.62:36761/ using document root <none> and password file <none>
I20260620 07:07:07.931185 13616 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:07.931246 13616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:07.931417 13616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:07.933310 13616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "bdd1ecd15afd4c9ea11ad9a08256979e"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d31ad4756770cb35f512e0b1fa7396d6"
server_key_iv: "3837c4e8612ca78193101909b81efd35"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.933799 13616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "bdd1ecd15afd4c9ea11ad9a08256979e"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d31ad4756770cb35f512e0b1fa7396d6"
server_key_iv: "3837c4e8612ca78193101909b81efd35"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.937125 13616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:07.939525 13630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:07.940658 13616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:07.940752 13616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "bdd1ecd15afd4c9ea11ad9a08256979e"
format_stamp: "Formatted at 2026-06-20 07:07:07 on dist-test-slave-wjw3"
server_key: "d31ad4756770cb35f512e0b1fa7396d6"
server_key_iv: "3837c4e8612ca78193101909b81efd35"
server_key_version: "encryptionkey@0"
I20260620 07:07:07.940834 13616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:07.952071 13616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:07.952632 13616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:07.952764 13616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:07.959714 13616 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38337
I20260620 07:07:07.959723 13682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38337 every 8 connection(s)
I20260620 07:07:07.960587 13616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:07.963575 13683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:07.968665 13683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e: Bootstrap starting.
I20260620 07:07:07.969209 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13616
I20260620 07:07:07.969305 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:07.969548 8288 external_mini_cluster.cc:1428] Setting key f930fe5f4d5ae11fdf38ca9bd059bcfc
I20260620 07:07:07.971236 13683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:07.971941 13683 log.cc:824] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:07.973659 13683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e: No bootstrap required, opened a new log
I20260620 07:07:07.977015 13683 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } }
I20260620 07:07:07.977248 13683 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:07.977339 13683 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd1ecd15afd4c9ea11ad9a08256979e, State: Initialized, Role: FOLLOWER
I20260620 07:07:07.977743 13683 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } }
I20260620 07:07:07.977859 13683 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:07.977916 13683 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:07.978000 13683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:07.978120 13686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:07.970740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.8.24.62:38337)
0620 07:07:07.971354 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:07.971372 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:07.971411 (+ 39us) server_negotiation.cc:407] Connection header received
0620 07:07:07.972057 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:07.972079 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:07.972473 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:07.972769 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:07.973175 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.974132 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.974986 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:07.975475 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:07.975617 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:07.976396 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:07.976401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:07.976412 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:07.976784 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:07.976837 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:07.976856 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:07.976934 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:07.977074 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 07:07:07.977331 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260620 07:07:07.978798 13683 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } }
I20260620 07:07:07.979130 13683 leader_election.cc:302] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd1ecd15afd4c9ea11ad9a08256979e; no voters:
I20260620 07:07:07.979409 13683 leader_election.cc:288] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:07.979508 13688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:07.979703 13688 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [term 1 LEADER]: Becoming Leader. State: Replica: bdd1ecd15afd4c9ea11ad9a08256979e, State: Running, Role: LEADER
I20260620 07:07:07.980026 13688 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } }
I20260620 07:07:07.980593 13683 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:07.980962 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:07:07.981565 13690 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd1ecd15afd4c9ea11ad9a08256979e. Latest consensus state: current_term: 1 leader_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } } }
I20260620 07:07:07.981912 13690 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:07.981817 13689 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd1ecd15afd4c9ea11ad9a08256979e" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38337 } } }
I20260620 07:07:07.982218 13689 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:07.983098 13699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:07.985646 13699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:07.990191 13699 catalog_manager.cc:1347] Generated new cluster ID: bb7ae1bd5ad9487188e89bc050f8b52a
I20260620 07:07:07.990248 13699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:08.007480 13699 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:08.008421 13699 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:08.012171 13699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd1ecd15afd4c9ea11ad9a08256979e: Generated new TSK 0
I20260620 07:07:08.012775 13699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:08.024046 13686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:08.019748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.8.24.62:38337)
0620 07:07:08.019864 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:08.019868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:08.020084 (+ 216us) server_negotiation.cc:407] Connection header received
0620 07:07:08.020229 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:08.020231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:08.020273 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:08.020352 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0620 07:07:08.020580 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.021096 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.021919 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.022132 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.022214 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:08.022490 (+ 276us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 07:07:08.023722 (+ 1232us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 07:07:08.023930 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 07:07:08.024097 13686 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 07:07:08.025913 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:08.019683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38337 (local address 127.0.0.1:39862)
0620 07:07:08.020002 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:08.020013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:08.020117 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:08.020362 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:08.020365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:08.020372 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 07:07:08.020482 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:08.020488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.021199 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.021203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:08.021815 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:08.021823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.022214 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.022236 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:08.022323 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:08.022344 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 07:07:08.023872 (+ 1528us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 07:07:08.025747 (+ 1875us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:38337: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40623/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":1696,"thread_start_us":88,"threads_started":1}
I20260620 07:07:08.026381 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13616
E20260620 07:07:08.034859 13713 webserver.cc:533] Webserver: sslize failed:
E20260620 07:07:08.034914 13713 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T07:07:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (267 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T07:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:08Z Disabled control of system clock
I20260620 07:07:08.071390 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35137
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35137
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/testchainca.cert with env {}
W20260620 07:07:08.173341 13717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:08.173574 13717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:08.173611 13717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:08.177983 13717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:08.178037 13717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:08.178052 13717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:08.178067 13717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:08.178087 13717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 07:07:08.178105 13717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 07:07:08.178120 13717 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 07:07:08.183102 13717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35137
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:08.184002 13717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:08.184863 13717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:08.192378 13722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:08.192437 13725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:08.192415 13723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:08.192687 13717 server_base.cc:1034] running on GCE node
I20260620 07:07:08.193186 13717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:08.194160 13717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:08.195308 13717 hybrid_clock.cc:648] HybridClock initialized: now 1781939228195292 us; error 31 us; skew 500 ppm
I20260620 07:07:08.196959 13717 webserver.cc:460] Webserver started at http://127.8.24.62:45259/ using document root <none> and password file <none>
I20260620 07:07:08.197461 13717 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:08.197525 13717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:08.197713 13717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:08.199584 13717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "196f1ef6de4c49259fd82c7d8519d6f5"
format_stamp: "Formatted at 2026-06-20 07:07:08 on dist-test-slave-wjw3"
server_key: "8d1e876f2863960a655b62dc4032c5f7"
server_key_iv: "30f9d1180f3b4fe754971e762952a1ae"
server_key_version: "encryptionkey@0"
I20260620 07:07:08.200121 13717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "196f1ef6de4c49259fd82c7d8519d6f5"
format_stamp: "Formatted at 2026-06-20 07:07:08 on dist-test-slave-wjw3"
server_key: "8d1e876f2863960a655b62dc4032c5f7"
server_key_iv: "30f9d1180f3b4fe754971e762952a1ae"
server_key_version: "encryptionkey@0"
I20260620 07:07:08.203308 13717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:08.205396 13731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:08.206466 13717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:08.206576 13717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "196f1ef6de4c49259fd82c7d8519d6f5"
format_stamp: "Formatted at 2026-06-20 07:07:08 on dist-test-slave-wjw3"
server_key: "8d1e876f2863960a655b62dc4032c5f7"
server_key_iv: "30f9d1180f3b4fe754971e762952a1ae"
server_key_version: "encryptionkey@0"
I20260620 07:07:08.206660 13717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:08.218345 13717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:08.218918 13717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:08.219071 13717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:08.225179 13717 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35137
I20260620 07:07:08.225189 13783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35137 every 8 connection(s)
I20260620 07:07:08.226033 13717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:08.228627 13784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:08.233703 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: Bootstrap starting.
I20260620 07:07:08.235641 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13717
I20260620 07:07:08.235751 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:08.236028 8288 external_mini_cluster.cc:1428] Setting key a734ad450249bc204f7148f66a18efdd
I20260620 07:07:08.236155 13784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:08.236918 13784 log.cc:824] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:08.238783 13784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: No bootstrap required, opened a new log
I20260620 07:07:08.242007 13784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } }
I20260620 07:07:08.242229 13784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:08.242287 13784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196f1ef6de4c49259fd82c7d8519d6f5, State: Initialized, Role: FOLLOWER
I20260620 07:07:08.242693 13784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } }
I20260620 07:07:08.242810 13784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:08.242867 13784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:08.242952 13784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:08.243829 13784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } }
I20260620 07:07:08.244130 13784 leader_election.cc:302] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 196f1ef6de4c49259fd82c7d8519d6f5; no voters:
I20260620 07:07:08.244372 13784 leader_election.cc:288] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:08.244465 13789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:08.244575 13788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:08.237070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.8.24.62:35137)
0620 07:07:08.237779 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:08.237799 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:08.237832 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:07:08.238442 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:08.238459 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:08.238775 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:08.239094 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:08.239536 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.240466 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.241296 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.241755 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.241902 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:08.242721 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:08.242727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:08.242742 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:08.243189 (+ 447us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:08.243243 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:08.243263 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:08.243345 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:08.243492 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 07:07:08.243739 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":99,"threads_started":1}
I20260620 07:07:08.244699 13789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [term 1 LEADER]: Becoming Leader. State: Replica: 196f1ef6de4c49259fd82c7d8519d6f5, State: Running, Role: LEADER
I20260620 07:07:08.245082 13789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } }
I20260620 07:07:08.245517 13784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:08.246627 13791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 196f1ef6de4c49259fd82c7d8519d6f5. Latest consensus state: current_term: 1 leader_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } } }
I20260620 07:07:08.246744 13791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:08.247035 13790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196f1ef6de4c49259fd82c7d8519d6f5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35137 } } }
I20260620 07:07:08.247162 13790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:08.247746 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 07:07:08.250092 13804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:08.250150 13804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:08.250221 13798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:08.250861 13798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:08.256196 13798 catalog_manager.cc:1347] Generated new cluster ID: c1c871adf3814ff390d903fa9c781c0a
I20260620 07:07:08.256255 13798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:08.275702 13798 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:08.276552 13798 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:08.283900 13798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 196f1ef6de4c49259fd82c7d8519d6f5: Generated new TSK 0
I20260620 07:07:08.284529 13798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:08.327731 13788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:08.293001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.8.24.62:35137)
0620 07:07:08.293121 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:08.293125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:08.293282 (+ 157us) server_negotiation.cc:407] Connection header received
0620 07:07:08.293425 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:08.293427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:08.293470 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:08.293551 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0620 07:07:08.293791 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.294313 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.295533 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.295743 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.295815 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:08.296091 (+ 276us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 07:07:08.327322 (+ 31231us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 07:07:08.327615 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54192: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 07:07:08.327807 13788 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54192: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 07:07:08.327980 13813 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:08.292928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35137 (local address 127.0.0.1:54192)
0620 07:07:08.293201 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:08.293215 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:08.293322 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:08.293575 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:08.293578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:08.293585 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 07:07:08.293703 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:08.293712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.294415 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.294418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:08.295406 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:08.295416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:08.295822 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:08.295843 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:08.295935 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:08.295962 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 07:07:08.327578 (+ 31616us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 07:07:08.327785 (+ 207us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.24.62:35137: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
I20260620 07:07:08.328570 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13717
2026-06-20T07:07:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (282 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (113 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (116 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:08 dist-test-slave-wjw3 krb5kdc[13825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:08 dist-test-slave-wjw3 krb5kdc[13825](info): set up 2 sockets
Jun 20 07:07:08 dist-test-slave-wjw3 krb5kdc[13825](info): commencing operation
krb5kdc: starting...
W20260620 07:07:09.511492 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:09 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:09.617779 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36153
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:09.720515 13838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:09.720744 13838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:09.720782 13838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:09.724612 13838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:09.724655 13838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:09.724673 13838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:09.724702 13838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:09.724730 13838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:09.729629 13838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:09.730516 13838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:09.731424 13838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:09.739058 13844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:09.739087 13843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:09.739161 13838 server_base.cc:1034] running on GCE node
W20260620 07:07:09.739053 13846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:09.739841 13838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:09.740839 13838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:09.741984 13838 hybrid_clock.cc:648] HybridClock initialized: now 1781939229741957 us; error 41 us; skew 500 ppm
Jun 20 07:07:09 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939229, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:09.744516 13838 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:09.745359 13838 webserver.cc:460] Webserver started at http://127.8.24.62:44985/ using document root <none> and password file <none>
I20260620 07:07:09.745877 13838 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:09.745939 13838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:09.746120 13838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:09.748142 13838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "f05171a445c746d8a12dfe80b406d94d"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "439b2d3039c12d3960450b04d72dc551"
server_key_iv: "5fdf9689bf844cb6a7232b61a0a394a8"
server_key_version: "encryptionkey@0"
I20260620 07:07:09.748672 13838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f05171a445c746d8a12dfe80b406d94d"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "439b2d3039c12d3960450b04d72dc551"
server_key_iv: "5fdf9689bf844cb6a7232b61a0a394a8"
server_key_version: "encryptionkey@0"
I20260620 07:07:09.752173 13838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:09.754343 13853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:09.755532 13838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:09.755640 13838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "f05171a445c746d8a12dfe80b406d94d"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "439b2d3039c12d3960450b04d72dc551"
server_key_iv: "5fdf9689bf844cb6a7232b61a0a394a8"
server_key_version: "encryptionkey@0"
I20260620 07:07:09.755723 13838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:09.779381 13838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:09.781980 13838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:09.782143 13838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:09.788650 13838 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:36153
I20260620 07:07:09.788655 13905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:36153 every 8 connection(s)
I20260620 07:07:09.789525 13838 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:09.792148 13906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:09.792436 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13838
I20260620 07:07:09.792544 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:09.792797 8288 external_mini_cluster.cc:1428] Setting key 69b1071a13eb07134a6f212efd07ef7b
I20260620 07:07:09.798146 13906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d: Bootstrap starting.
I20260620 07:07:09.800228 13906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d: Neither blocks nor log segments found. Creating new log.
Jun 20 07:07:09 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:09.801046 13906 log.cc:824] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:09.803074 13906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d: No bootstrap required, opened a new log
I20260620 07:07:09.805987 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:09.794091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.8.24.62:36153)
0620 07:07:09.794509 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:09.794526 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:09.794558 (+ 32us) server_negotiation.cc:407] Connection header received
0620 07:07:09.795213 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:09.795229 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:09.795578 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:09.795873 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:09.796608 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:09.797557 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:09.798401 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:09.798851 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:09.798991 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:09.801425 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:09.801430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:09.801442 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:09.801467 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:09.803172 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:09.803569 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:09.803575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:09.803580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:09.803714 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:09.803937 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:09.803940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:09.803942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:09.804415 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:09.804595 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:09.804759 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 07:07:09.805058 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":130,"threads_started":1}
I20260620 07:07:09.807322 13906 raft_consensus.cc:348] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } }
I20260620 07:07:09.807575 13906 raft_consensus.cc:374] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:09.807633 13906 raft_consensus.cc:729] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05171a445c746d8a12dfe80b406d94d, State: Initialized, Role: FOLLOWER
I20260620 07:07:09.808072 13906 consensus_queue.cc:260] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } }
I20260620 07:07:09.808223 13906 raft_consensus.cc:388] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:09.808277 13906 raft_consensus.cc:482] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:09.808344 13906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:09.809111 13906 raft_consensus.cc:504] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } }
I20260620 07:07:09.809386 13906 leader_election.cc:302] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f05171a445c746d8a12dfe80b406d94d; no voters:
I20260620 07:07:09.809618 13906 leader_election.cc:288] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:09.809705 13911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:09.809876 13911 raft_consensus.cc:686] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [term 1 LEADER]: Becoming Leader. State: Replica: f05171a445c746d8a12dfe80b406d94d, State: Running, Role: LEADER
I20260620 07:07:09.810179 13911 consensus_queue.cc:237] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } }
I20260620 07:07:09.810714 13906 sys_catalog.cc:564] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:09.811704 13912 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f05171a445c746d8a12dfe80b406d94d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } } }
I20260620 07:07:09.811722 13913 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f05171a445c746d8a12dfe80b406d94d. Latest consensus state: current_term: 1 leader_uuid: "f05171a445c746d8a12dfe80b406d94d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05171a445c746d8a12dfe80b406d94d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36153 } } }
I20260620 07:07:09.811905 13913 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:09.811899 13912 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:09.812616 13920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:09.815639 13920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:09.820814 13920 catalog_manager.cc:1347] Generated new cluster ID: 25abc27ec58e468a8885d7502c9e2e84
I20260620 07:07:09.820885 13920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:09.840286 13920 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:09.841418 13920 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:09.844821 13920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f05171a445c746d8a12dfe80b406d94d: Generated new TSK 0
I20260620 07:07:09.845403 13920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:09.873929 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:09.978617 13932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:09.978853 13932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:09.978911 13932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:09.982834 13932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:09.982887 13932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:09.982960 13932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:09.987998 13932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:09.988965 13932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:09.989931 13932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:09.998693 13937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:09.998756 13940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:09.998764 13938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:09.998883 13932 server_base.cc:1034] running on GCE node
I20260620 07:07:09.999289 13932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:09.999864 13932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:10.001008 13932 hybrid_clock.cc:648] HybridClock initialized: now 1781939230000992 us; error 26 us; skew 500 ppm
Jun 20 07:07:09 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939229, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.003553 13932 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:10.004315 13932 webserver.cc:460] Webserver started at http://127.8.24.1:44395/ using document root <none> and password file <none>
I20260620 07:07:10.004812 13932 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:10.004871 13932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:10.005059 13932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:10.006974 13932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "dd89e7ffcfe24a5d9e81985e46d1aa83"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "7ce568d5e71675860be3bb940d11f662"
server_key_iv: "ff964d0ef0b6aa4c7cb39e32738d1eef"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.007534 13932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "dd89e7ffcfe24a5d9e81985e46d1aa83"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "7ce568d5e71675860be3bb940d11f662"
server_key_iv: "ff964d0ef0b6aa4c7cb39e32738d1eef"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.010903 13932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:07:10.013264 13947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.014405 13932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:10.014494 13932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "dd89e7ffcfe24a5d9e81985e46d1aa83"
format_stamp: "Formatted at 2026-06-20 07:07:09 on dist-test-slave-wjw3"
server_key: "7ce568d5e71675860be3bb940d11f662"
server_key_iv: "ff964d0ef0b6aa4c7cb39e32738d1eef"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.014582 13932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:10.023499 13932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:10.026292 13932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:10.026427 13932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:10.026911 13932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:10.027670 13932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:10.027715 13932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.027752 13932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:10.027771 13932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.035755 13932 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43151
I20260620 07:07:10.035786 14060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43151 every 8 connection(s)
I20260620 07:07:10.036569 13932 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:10.039006 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 13932
I20260620 07:07:10.039130 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:10.039458 8288 external_mini_cluster.cc:1428] Setting key 56cf42ffcd3c5fac21c991be273bdc48
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939229, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:07:10.049536 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.038173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58239 (local address 127.8.24.62:36153)
0620 07:07:10.038294 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.038297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.039176 (+ 879us) server_negotiation.cc:407] Connection header received
0620 07:07:10.040711 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.040714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.040765 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.040834 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.041944 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.042473 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.043610 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.043834 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.043920 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.046351 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.046354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.046356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.046379 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.047795 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.048219 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.048221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.048222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.048302 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.048577 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.048580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.048581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.048748 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.048821 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.049271 (+ 450us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.049375 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:10.050331 14063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.038528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36153 (local address 127.8.24.1:58239)
0620 07:07:10.039016 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:10.039074 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:10.040501 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:10.041002 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:10.041009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:10.041343 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:10.041807 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.041819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.042590 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.042593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.043491 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.043501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.043964 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.043969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.044129 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.044680 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:10.044695 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:10.046185 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:10.047891 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.047896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.047907 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.048145 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.048381 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.048383 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.048384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.048511 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.048819 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:10.048823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:10.049078 (+ 255us) client_negotiation.cc:769] Sending connection context
0620 07:07:10.049229 (+ 151us) client_negotiation.cc:240] Negotiation successful
0620 07:07:10.049468 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":113,"threads_started":1}
I20260620 07:07:10.052126 14061 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36153
I20260620 07:07:10.052376 14061 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:10.052927 14061 heartbeater.cc:511] Master 127.8.24.62:36153 requested a full tablet report, sending...
I20260620 07:07:10.054448 13870 ts_manager.cc:194] Registered new tserver with Master: dd89e7ffcfe24a5d9e81985e46d1aa83 (127.8.24.1:43151)
I20260620 07:07:10.055552 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:58239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:10.063450 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:10.168054 14066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:10.168289 14066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:10.168334 14066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:10.172230 14066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:10.172279 14066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:10.172348 14066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:10.177376 14066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:10.178368 14066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:10.179318 14066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:10.188318 14074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:10.188344 14066 server_base.cc:1034] running on GCE node
W20260620 07:07:10.188354 14071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:10.188371 14072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:10.188951 14066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:10.189538 14066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:10.190727 14066 hybrid_clock.cc:648] HybridClock initialized: now 1781939230190703 us; error 31 us; skew 500 ppm
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.193455 14066 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:10.194275 14066 webserver.cc:460] Webserver started at http://127.8.24.2:45159/ using document root <none> and password file <none>
I20260620 07:07:10.194819 14066 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:10.194882 14066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:10.195067 14066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:10.197178 14066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "b82b7cf824594db88410818723ab4351"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "9f918f587e3d0b74bb340f2808aa8c9a"
server_key_iv: "6f2da9446dbb51a8bc9db389e60923d4"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.197717 14066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "b82b7cf824594db88410818723ab4351"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "9f918f587e3d0b74bb340f2808aa8c9a"
server_key_iv: "6f2da9446dbb51a8bc9db389e60923d4"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.201269 14066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 07:07:10.203631 14081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.204800 14066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:10.204900 14066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "b82b7cf824594db88410818723ab4351"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "9f918f587e3d0b74bb340f2808aa8c9a"
server_key_iv: "6f2da9446dbb51a8bc9db389e60923d4"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.204990 14066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:10.218184 14066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:10.221205 14066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:10.221366 14066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:10.221954 14066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:10.222715 14066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:10.222760 14066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.222812 14066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:10.222838 14066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.230691 14066 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38761
I20260620 07:07:10.230741 14194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38761 every 8 connection(s)
I20260620 07:07:10.231551 14066 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:10.238843 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14066
I20260620 07:07:10.238924 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:10.239176 8288 external_mini_cluster.cc:1428] Setting key b5bba5725417215e911e25022280a6b0
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:10.244012 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.233195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34219 (local address 127.8.24.62:36153)
0620 07:07:10.233314 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.233318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.234122 (+ 804us) server_negotiation.cc:407] Connection header received
0620 07:07:10.235411 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.235414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.235480 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.235550 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.236587 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.237104 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.238155 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.238356 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.238419 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.240748 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.240751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.240753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.240776 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.242008 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.242531 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.242534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.242535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.242600 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.242928 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.242931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.242932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.243097 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.243181 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.243725 (+ 544us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.243846 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:10.244760 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.233552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36153 (local address 127.8.24.2:34219)
0620 07:07:10.233973 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:10.234013 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:10.235228 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:10.235683 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:10.235689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:10.236013 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:10.236482 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.236495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.237187 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.237189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.238070 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.238078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.238472 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.238474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.238605 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.239155 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:10.239170 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:10.240596 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:10.242112 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.242118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.242132 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.242447 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.242671 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.242674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.242676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.242846 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.243206 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:10.243212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:10.243507 (+ 295us) client_negotiation.cc:769] Sending connection context
0620 07:07:10.243673 (+ 166us) client_negotiation.cc:240] Negotiation successful
0620 07:07:10.243969 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260620 07:07:10.245903 14195 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36153
I20260620 07:07:10.246110 14195 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:10.246556 14195 heartbeater.cc:511] Master 127.8.24.62:36153 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:07:10.247594 13870 ts_manager.cc:194] Registered new tserver with Master: b82b7cf824594db88410818723ab4351 (127.8.24.2:38761)
I20260620 07:07:10.248108 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:34219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:10.261915 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:10.366405 14200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:10.366636 14200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:10.366674 14200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:10.370558 14200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:10.370608 14200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:10.370662 14200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:10.375653 14200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:10.376631 14200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:10.377637 14200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:10.386353 14208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:10.386353 14205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:10.386410 14206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:10.386557 14200 server_base.cc:1034] running on GCE node
I20260620 07:07:10.386956 14200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:10.387544 14200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:10.388687 14200 hybrid_clock.cc:648] HybridClock initialized: now 1781939230388670 us; error 29 us; skew 500 ppm
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.391100 14200 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:10.391891 14200 webserver.cc:460] Webserver started at http://127.8.24.3:42631/ using document root <none> and password file <none>
I20260620 07:07:10.392359 14200 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:10.392411 14200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:10.392566 14200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:10.394397 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "72be5447926248b4a13fea33bbaf09fb"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "3b90b4becdbe5083509f6334098c3c38"
server_key_iv: "4112ecba8f17160d765accdc6f22d26b"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.394893 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "72be5447926248b4a13fea33bbaf09fb"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "3b90b4becdbe5083509f6334098c3c38"
server_key_iv: "4112ecba8f17160d765accdc6f22d26b"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.398249 14200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:07:10.400367 14215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.401432 14200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:10.401527 14200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "72be5447926248b4a13fea33bbaf09fb"
format_stamp: "Formatted at 2026-06-20 07:07:10 on dist-test-slave-wjw3"
server_key: "3b90b4becdbe5083509f6334098c3c38"
server_key_iv: "4112ecba8f17160d765accdc6f22d26b"
server_key_version: "encryptionkey@0"
I20260620 07:07:10.401611 14200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:10.418385 14200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:10.420989 14200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:10.421156 14200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:10.421679 14200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:10.422425 14200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:10.422470 14200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.422520 14200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:10.422549 14200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:10.430593 14200 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:39823
I20260620 07:07:10.430620 14328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:39823 every 8 connection(s)
I20260620 07:07:10.431412 14200 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:10.437244 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14200
I20260620 07:07:10.437350 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:10.437595 8288 external_mini_cluster.cc:1428] Setting key 11ba9e94e7947aa97ab5491e23a61612
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:10.443717 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.433181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44169 (local address 127.8.24.62:36153)
0620 07:07:10.433347 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.433351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.434162 (+ 811us) server_negotiation.cc:407] Connection header received
0620 07:07:10.435412 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.435415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.435488 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.435554 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.436615 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.437197 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.438408 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.438666 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.438763 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.441000 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.441003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.441005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.441026 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.442210 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.442595 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.442597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.442598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.442661 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.442889 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.442890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.442891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.443042 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.443110 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.443500 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.443598 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 07:07:10.444422 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.433587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:36153 (local address 127.8.24.3:44169)
0620 07:07:10.434011 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:10.434053 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:10.435236 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:10.435714 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:10.435721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:10.436048 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:10.436511 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.436524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.437294 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.437297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.438298 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:10.438308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.438820 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.438824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:10.438988 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.439560 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:10.439579 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:10.440863 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:10.442297 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.442301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.442311 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.442527 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.442719 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:10.442721 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:10.442722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:10.442830 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:10.443106 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:10.443110 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:10.443308 (+ 198us) client_negotiation.cc:769] Sending connection context
0620 07:07:10.443460 (+ 152us) client_negotiation.cc:240] Negotiation successful
0620 07:07:10.443684 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260620 07:07:10.445379 14329 heartbeater.cc:347] Connected to a master server at 127.8.24.62:36153
I20260620 07:07:10.445570 14329 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:10.445982 14329 heartbeater.cc:511] Master 127.8.24.62:36153 requested a full tablet report, sending...
I20260620 07:07:10.446862 13870 ts_manager.cc:194] Registered new tserver with Master: 72be5447926248b4a13fea33bbaf09fb (127.8.24.3:39823)
I20260620 07:07:10.447376 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:44169
I20260620 07:07:10.450945 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/test-admin.keytab.
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.463699 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 07:07:10.466229 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:10.464927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.8.24.1:43151)
0620 07:07:10.465273 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.465283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.465305 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:07:10.465351 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.465356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.465534 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.465675 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.466078 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40276: BlockingRecv error: recv got EOF from 127.0.0.1:40276 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":154,"threads_started":1}
W20260620 07:07:10.466542 8288 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc' of type FILE
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.467423 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.468230 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:07:10.476679 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.468990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.8.24.1:43151)
0620 07:07:10.469103 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.469107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.469177 (+ 70us) server_negotiation.cc:407] Connection header received
0620 07:07:10.469267 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.469269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.469304 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.469366 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.469793 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.470555 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.471286 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.471684 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.471758 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.473801 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.473810 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.473822 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.473846 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.475150 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.475556 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.475561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.475563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.475636 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.475835 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.475837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.475838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.476091 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.476240 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.476374 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.476506 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 07:07:10.480027 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:10.479258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.8.24.1:43151)
0620 07:07:10.479378 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.479381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.479420 (+ 39us) server_negotiation.cc:407] Connection header received
0620 07:07:10.479534 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.479537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.479578 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.479635 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.479936 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40296: BlockingRecv error: recv got EOF from 127.0.0.1:40296 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 07:07:10.480108 8288 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc' of type FILE
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.480991 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.481719 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:07:10.489039 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.482498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.8.24.1:43151)
0620 07:07:10.482616 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.482618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.482666 (+ 48us) server_negotiation.cc:407] Connection header received
0620 07:07:10.482751 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.482753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.482784 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.482842 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.483227 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.483798 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.484497 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.484707 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.484770 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.486642 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.486645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.486647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.486667 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.487898 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.488206 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.488209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.488210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.488270 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.488441 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.488443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.488444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.488589 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.488675 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.488815 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.488894 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 07:07:10.491303 14336 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:10.490475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.8.24.1:43151)
0620 07:07:10.490588 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.490591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.490671 (+ 80us) server_negotiation.cc:407] Connection header received
0620 07:07:10.490778 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.490781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.490818 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.490875 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.491216 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40306: BlockingRecv error: recv got EOF from 127.0.0.1:40306 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 07:07:10.491372 8288 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781939200335300-8288-0/krb5kdc/krb5cc' of type FILE
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.492300 8288 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:10.493037 8288 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[13825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:07:10.500195 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:10.493752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.8.24.1:43151)
0620 07:07:10.493864 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:10.493867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:10.493908 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:07:10.494000 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:10.494003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:10.494034 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:10.494097 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:10.494482 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.494945 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.495680 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:10.495887 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:10.495957 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:10.497877 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:10.497880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:10.497882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:10.497904 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:10.498942 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.499272 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.499275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.499276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.499338 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:10.499561 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:10.499563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:10.499564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:10.499708 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:10.499807 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:10.499951 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 07:07:10.500032 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:07:10.501086 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13932
I20260620 07:07:10.505971 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14066
I20260620 07:07:10.510005 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14200
I20260620 07:07:10.514142 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 13838
2026-06-20T07:07:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1957 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[14349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[14349](info): set up 2 sockets
Jun 20 07:07:10 dist-test-slave-wjw3 krb5kdc[14349](info): commencing operation
krb5kdc: starting...
W20260620 07:07:11.467491 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:11 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
I20260620 07:07:11.576212 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42557
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:11.679258 14362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:11.679522 14362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:11.679577 14362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:11.683463 14362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:11.683512 14362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:11.683530 14362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:11.683563 14362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:11.683591 14362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:11.688618 14362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:11.689579 14362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:11.690526 14362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:11.698275 14368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:11.698334 14362 server_base.cc:1034] running on GCE node
W20260620 07:07:11.698256 14367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:11.698274 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:11.698977 14362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:11.700064 14362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:11.701217 14362 hybrid_clock.cc:648] HybridClock initialized: now 1781939231701194 us; error 35 us; skew 500 ppm
Jun 20 07:07:11 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:11.703671 14362 init.cc:377] Logged in from keytab as oryx/127.8.24.62@KRBTEST.COM (short username oryx)
I20260620 07:07:11.704480 14362 webserver.cc:460] Webserver started at http://127.8.24.62:34679/ using document root <none> and password file <none>
I20260620 07:07:11.704988 14362 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:11.705049 14362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:11.705228 14362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:11.707173 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "f277a978387143d89c960def5b77cf24"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "0afa7ae4f04f8eed3195f1acc826bb95"
server_key_iv: "1f7b0a389f6e78b77cb7b7c2568bd733"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.707712 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f277a978387143d89c960def5b77cf24"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "0afa7ae4f04f8eed3195f1acc826bb95"
server_key_iv: "1f7b0a389f6e78b77cb7b7c2568bd733"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.711062 14362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:07:11.713268 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:11.714402 14362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:11.714509 14362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "f277a978387143d89c960def5b77cf24"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "0afa7ae4f04f8eed3195f1acc826bb95"
server_key_iv: "1f7b0a389f6e78b77cb7b7c2568bd733"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.714605 14362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:11.730492 14362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:11.733206 14362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:11.733366 14362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:11.739692 14362 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:42557
I20260620 07:07:11.739928 14429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:42557 every 8 connection(s)
I20260620 07:07:11.740585 14362 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:11.740840 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14362
I20260620 07:07:11.740942 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:11.741237 8288 external_mini_cluster.cc:1428] Setting key 20d050ceda65a4c71bbfdb86e20c91bf
I20260620 07:07:11.743655 14430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:11.749686 14430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Bootstrap starting.
Jun 20 07:07:11 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:11.752322 14430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:11.753175 14430 log.cc:824] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:11.755093 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:11.742515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.8.24.62:42557)
0620 07:07:11.742895 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:11.742912 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:11.742952 (+ 40us) server_negotiation.cc:407] Connection header received
0620 07:07:11.743783 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:11.743805 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:11.744198 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:11.744536 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:11.745140 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.746583 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.747342 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.748037 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.748198 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:11.750325 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:11.750331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:11.750346 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:11.750375 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:11.752193 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:11.752599 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:11.752604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:11.752610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:11.752743 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:11.752967 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:11.752970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:11.752972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:11.753472 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:11.753648 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:11.753899 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 07:07:11.754178 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":95,"threads_started":1}
I20260620 07:07:11.755357 14430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: No bootstrap required, opened a new log
I20260620 07:07:11.758922 14430 raft_consensus.cc:348] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } }
I20260620 07:07:11.759145 14430 raft_consensus.cc:374] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:11.759186 14430 raft_consensus.cc:729] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f277a978387143d89c960def5b77cf24, State: Initialized, Role: FOLLOWER
I20260620 07:07:11.759565 14430 consensus_queue.cc:260] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } }
I20260620 07:07:11.759683 14430 raft_consensus.cc:388] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:11.759727 14430 raft_consensus.cc:482] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:11.759795 14430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:11.760598 14430 raft_consensus.cc:504] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } }
I20260620 07:07:11.760912 14430 leader_election.cc:302] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f277a978387143d89c960def5b77cf24; no voters:
I20260620 07:07:11.761227 14430 leader_election.cc:288] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:11.761307 14435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:11.761492 14435 raft_consensus.cc:686] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 LEADER]: Becoming Leader. State: Replica: f277a978387143d89c960def5b77cf24, State: Running, Role: LEADER
I20260620 07:07:11.761796 14435 consensus_queue.cc:237] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } }
I20260620 07:07:11.762326 14430 sys_catalog.cc:564] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:11.763245 14437 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f277a978387143d89c960def5b77cf24. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } }
I20260620 07:07:11.763363 14437 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:11.763711 14444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:11.763718 14436 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } }
I20260620 07:07:11.763909 14436 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:11.766289 14444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:11.770944 14444 catalog_manager.cc:1347] Generated new cluster ID: 45ad0780c6dc4b90b9329ca783f0f075
I20260620 07:07:11.770991 14444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:11.781490 14444 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:11.782264 14444 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:11.789563 14444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Generated new TSK 0
I20260620 07:07:11.790131 14444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
I20260620 07:07:11.815840 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:11.919294 14456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:11.919562 14456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:11.919613 14456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:11.923513 14456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:11.923564 14456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:11.923624 14456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:11.928597 14456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:11.929494 14456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:11.930437 14456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:11.939076 14464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:11.939064 14461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:11.939131 14462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:11.939261 14456 server_base.cc:1034] running on GCE node
I20260620 07:07:11.939735 14456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:11.940316 14456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:11.941470 14456 hybrid_clock.cc:648] HybridClock initialized: now 1781939231941449 us; error 32 us; skew 500 ppm
Jun 20 07:07:11 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:11.943953 14456 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260620 07:07:11.944751 14456 webserver.cc:460] Webserver started at http://127.8.24.1:40461/ using document root <none> and password file <none>
I20260620 07:07:11.945264 14456 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:11.945328 14456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:11.945505 14456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:11.947464 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "45c3a263a49b440990db588277f98150"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "733fbcc19badd65ce69d11e6c20c3af3"
server_key_iv: "0927afe0d40e944da060a28fe91c93d2"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.948002 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "45c3a263a49b440990db588277f98150"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "733fbcc19badd65ce69d11e6c20c3af3"
server_key_iv: "0927afe0d40e944da060a28fe91c93d2"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.951560 14456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:11.953824 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:11.954964 14456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:11.955072 14456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "45c3a263a49b440990db588277f98150"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "733fbcc19badd65ce69d11e6c20c3af3"
server_key_iv: "0927afe0d40e944da060a28fe91c93d2"
server_key_version: "encryptionkey@0"
I20260620 07:07:11.955161 14456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:11.973145 14456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:11.976128 14456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:11.976285 14456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:11.976823 14456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:11.977555 14456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:11.977607 14456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:11.977651 14456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:11.977679 14456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:11.985342 14456 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37015
I20260620 07:07:11.985390 14584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37015 every 8 connection(s)
I20260620 07:07:11.986158 14456 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:11.991179 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14456
I20260620 07:07:11.991276 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:11.991544 8288 external_mini_cluster.cc:1428] Setting key 591596ebb187fc76ccb73bcce82610d9
Jun 20 07:07:11 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:11.999063 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:11.987774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33851 (local address 127.8.24.62:42557)
0620 07:07:11.987898 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:11.987902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:11.988880 (+ 978us) server_negotiation.cc:407] Connection header received
0620 07:07:11.989742 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:11.989745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:11.989802 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:11.989873 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:11.990981 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.991630 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.992718 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.992999 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.993106 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:11.995562 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:11.995565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:11.995568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:11.995595 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:11.997008 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:11.997465 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:11.997469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:11.997471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:11.997575 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:11.997937 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:11.997940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:11.997942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:11.998163 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:11.998265 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:11.998767 (+ 502us) server_negotiation.cc:299] Negotiation successful
0620 07:07:11.998899 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:07:11.999830 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:11.988133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:33851)
0620 07:07:11.988742 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:11.988782 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:11.989535 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:11.990043 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:11.990050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:11.990381 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:11.990841 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:11.990854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.991742 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.991745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:11.992604 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:11.992613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:11.993159 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:11.993164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:11.993349 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:11.993916 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:11.993935 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:11.995385 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:11.997126 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:11.997130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:11.997141 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:11.997374 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:11.997688 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:11.997691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:11.997692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:11.997841 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:11.998269 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:11.998275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:11.998539 (+ 264us) client_negotiation.cc:769] Sending connection context
0620 07:07:11.998696 (+ 157us) client_negotiation.cc:240] Negotiation successful
0620 07:07:11.998947 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:07:12.000974 14585 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:12.001176 14585 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:12.001634 14585 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:12.003105 14394 ts_manager.cc:194] Registered new tserver with Master: 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
I20260620 07:07:12.004223 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:33851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
I20260620 07:07:12.015393 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:12.120121 14590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:12.120373 14590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:12.120441 14590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:12.124323 14590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:12.124374 14590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:12.124456 14590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:12.129524 14590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:12.130528 14590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:12.131520 14590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:12.140811 14595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.140759 14598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.140763 14596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:12.141163 14590 server_base.cc:1034] running on GCE node
I20260620 07:07:12.141561 14590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:12.142161 14590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:12.143312 14590 hybrid_clock.cc:648] HybridClock initialized: now 1781939232143276 us; error 49 us; skew 500 ppm
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:12.149873 14590 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260620 07:07:12.150674 14590 webserver.cc:460] Webserver started at http://127.8.24.2:39387/ using document root <none> and password file <none>
I20260620 07:07:12.151165 14590 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.151217 14590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:12.151379 14590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:12.153343 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "5e8c643e69c246ed8a84dd255c133453"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "710cc24431a283b52af5f2c4cc49ed19"
server_key_iv: "a3c81f4c53147a611025e00116908368"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.153837 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "5e8c643e69c246ed8a84dd255c133453"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "710cc24431a283b52af5f2c4cc49ed19"
server_key_iv: "a3c81f4c53147a611025e00116908368"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.157311 14590 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:12.159590 14605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.160746 14590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:12.160837 14590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "5e8c643e69c246ed8a84dd255c133453"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "710cc24431a283b52af5f2c4cc49ed19"
server_key_iv: "a3c81f4c53147a611025e00116908368"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.160933 14590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.178844 14590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:12.181394 14590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:12.181531 14590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:12.182000 14590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:12.182739 14590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:12.182781 14590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.182816 14590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:12.182837 14590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.190682 14590 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45503
I20260620 07:07:12.190707 14718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45503 every 8 connection(s)
I20260620 07:07:12.191529 14590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:12.200441 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14590
I20260620 07:07:12.200539 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:12.200826 8288 external_mini_cluster.cc:1428] Setting key 5b26e86e1b88a99f00dfd8eee663c733
I20260620 07:07:12.203768 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.193192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:60145 (local address 127.8.24.62:42557)
0620 07:07:12.193317 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.193321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.194304 (+ 983us) server_negotiation.cc:407] Connection header received
0620 07:07:12.195104 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.195107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.195152 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.195220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.196301 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.196809 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.197848 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.198052 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.198119 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.200349 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.200353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.200355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.200380 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.201875 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.202382 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.202386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.202387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.202476 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.202756 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.202759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.202761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.202965 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.203052 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.203504 (+ 452us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.203617 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:07:12.204689 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.193559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:60145)
0620 07:07:12.194158 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.194196 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.194938 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.195359 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.195365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.195743 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.196195 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.196207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.196894 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.196896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.197759 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.197768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.198154 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.198156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.198286 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.198776 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.198791 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.200167 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.201979 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.201985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.202000 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.202293 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.202556 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.202558 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.202559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.202678 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.203060 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.203064 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.203280 (+ 216us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.203424 (+ 144us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.203699 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":107,"threads_started":1}
I20260620 07:07:12.205749 14719 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:12.205950 14719 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:12.206403 14719 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
I20260620 07:07:12.207389 14394 ts_manager.cc:194] Registered new tserver with Master: 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503)
I20260620 07:07:12.207930 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:60145
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260620 07:07:12.224116 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:12.328990 14724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:12.329231 14724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:12.329270 14724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:12.333127 14724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:12.333181 14724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:12.333236 14724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:12.338307 14724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:12.339224 14724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:12.340185 14724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:12.349155 14732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.349165 14730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.349175 14729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:12.349489 14724 server_base.cc:1034] running on GCE node
I20260620 07:07:12.349858 14724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:12.350417 14724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:12.351559 14724 hybrid_clock.cc:648] HybridClock initialized: now 1781939232351544 us; error 28 us; skew 500 ppm
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:12.353957 14724 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260620 07:07:12.354707 14724 webserver.cc:460] Webserver started at http://127.8.24.3:36791/ using document root <none> and password file <none>
I20260620 07:07:12.355185 14724 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.355237 14724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:12.355391 14724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:12.357275 14724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "ee8d07b71c6a4db69f08fd2eace1f92c"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "035c812299dd7fafa9acd3d30bd024b5"
server_key_iv: "8f23b229cdf0f74301ddbc58fe391f5b"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.357762 14724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "ee8d07b71c6a4db69f08fd2eace1f92c"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "035c812299dd7fafa9acd3d30bd024b5"
server_key_iv: "8f23b229cdf0f74301ddbc58fe391f5b"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.361215 14724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:12.363297 14739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.364571 14724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:12.364663 14724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "ee8d07b71c6a4db69f08fd2eace1f92c"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "035c812299dd7fafa9acd3d30bd024b5"
server_key_iv: "8f23b229cdf0f74301ddbc58fe391f5b"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.364739 14724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.376984 14724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:12.379407 14724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:12.379581 14724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:12.380061 14724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:12.380772 14724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:12.380859 14724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.380919 14724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:12.380949 14724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.390499 14724 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36375
I20260620 07:07:12.390518 14852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36375 every 8 connection(s)
I20260620 07:07:12.391587 14724 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:12.400135 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14724
I20260620 07:07:12.400241 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:12.400507 8288 external_mini_cluster.cc:1428] Setting key 2976ab08b3f755858386f9f921fa0e9f
I20260620 07:07:12.403638 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.393211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49933 (local address 127.8.24.62:42557)
0620 07:07:12.393340 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.393344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.394283 (+ 939us) server_negotiation.cc:407] Connection header received
0620 07:07:12.395082 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.395084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.395128 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.395196 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.396303 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.396819 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.397841 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.398038 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.398101 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.400434 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.400437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.400439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.400464 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.401858 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.402312 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.402315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.402317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.402408 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.402721 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.402723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.402724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.402897 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.402967 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.403382 (+ 415us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.403519 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:12.404414 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.393563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:49933)
0620 07:07:12.394149 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.394188 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.394921 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.395340 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.395346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.395722 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.396184 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.396197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.396899 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.396902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.397756 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.397765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.398138 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.398141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.398272 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.398797 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.398813 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.400268 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.401960 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.401965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.401977 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.402225 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.402497 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.402500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.402502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.402652 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.402968 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.402973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.403206 (+ 233us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.403351 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.403636 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":100,"threads_started":1}
I20260620 07:07:12.405401 14853 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:12.405625 14853 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:12.406075 14853 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
I20260620 07:07:12.406967 14394 ts_manager.cc:194] Registered new tserver with Master: ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:12.407590 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:49933
I20260620 07:07:12.413859 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.61.keytab.
Entry for principal oryx/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.61.keytab.
I20260620 07:07:12.436434 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:39679
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.61.keytab
--principal=oryx/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42557,127.8.24.61:39679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:12.539701 14858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:12.539942 14858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:12.539980 14858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:12.543927 14858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:12.543979 14858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:12.543996 14858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:12.544021 14858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:12.544036 14858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:12.549118 14858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42557,127.8.24.61:39679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.61.keytab
--principal=oryx/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:39679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
W20260620 07:07:12.549283 14858 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.24.62:42557,127.8.24.61:39679'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 07:07:12.550048 14858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:12.551005 14858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:12.558799 14864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.558811 14863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.558815 14866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:12.559026 14858 server_base.cc:1034] running on GCE node
I20260620 07:07:12.559583 14858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:12.560636 14858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:12.561815 14858 hybrid_clock.cc:648] HybridClock initialized: now 1781939232561796 us; error 33 us; skew 500 ppm
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:12.564553 14858 init.cc:377] Logged in from keytab as oryx/127.8.24.61@KRBTEST.COM (short username oryx)
I20260620 07:07:12.565455 14858 webserver.cc:460] Webserver started at http://127.8.24.61:39741/ using document root <none> and password file <none>
I20260620 07:07:12.565960 14858 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.566015 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:12.566176 14858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:12.568177 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/instance:
uuid: "995e53db55434dc9938a2720cc84e872"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "33c434dad4e8fa8109284f077591ac86"
server_key_iv: "c6edecabe0f2ba8ec3f1dea2f8b11955"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.568651 14858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal/instance:
uuid: "995e53db55434dc9938a2720cc84e872"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "33c434dad4e8fa8109284f077591ac86"
server_key_iv: "c6edecabe0f2ba8ec3f1dea2f8b11955"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.572057 14858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:12.574266 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.575398 14858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:12.575517 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "995e53db55434dc9938a2720cc84e872"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "33c434dad4e8fa8109284f077591ac86"
server_key_iv: "c6edecabe0f2ba8ec3f1dea2f8b11955"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.575600 14858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.595533 14858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:12.598091 14858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:12.598232 14858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:12.604648 14858 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:39679
I20260620 07:07:12.604894 14925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:39679 every 8 connection(s)
I20260620 07:07:12.605544 14858 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
I20260620 07:07:12.608173 14926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:12.611101 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14858
I20260620 07:07:12.611820 14926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260620 07:07:12.622890 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.613678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.8.24.62:42557)
0620 07:07:12.613789 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.613793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.614077 (+ 284us) server_negotiation.cc:407] Connection header received
0620 07:07:12.614382 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.614385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.614438 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.614505 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.615263 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.615823 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.616781 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.616991 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.617085 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.619649 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.619652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.619655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.619681 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.620923 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.621415 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.621418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.621420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.621500 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.621790 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.621792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.621794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.622015 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.622119 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.622643 (+ 524us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.622759 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:07:12.623790 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.613632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51426)
0620 07:07:12.613950 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.613986 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.614207 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.614568 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.614573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.614886 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.615125 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.615137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.615936 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.615938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.616668 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.616676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.617118 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.617121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.617316 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.617880 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.617900 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.619517 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.621018 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.621025 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.621034 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.621330 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.621576 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.621579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.621581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.621704 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.622119 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.622124 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.622403 (+ 279us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.622567 (+ 164us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.622836 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":3,"spinlock_wait_cycles":15360,"thread_start_us":96,"threads_started":1}
I20260620 07:07:12.623788 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.612663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.8.24.61:39679)
0620 07:07:12.613063 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.613076 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.613107 (+ 31us) server_negotiation.cc:407] Connection header received
0620 07:07:12.613877 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.613898 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.614218 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.614433 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.615203 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.616114 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.616887 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.617280 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.617373 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.620158 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.620162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.620171 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.620192 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.621656 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.622032 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.622037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.622039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.622118 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.622356 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.622358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.622359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.622565 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.622737 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.622959 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.623088 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":2944,"thread_start_us":119,"threads_started":1}
I20260620 07:07:12.624989 14926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260620 07:07:12.631944 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.625409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53392)
0620 07:07:12.625504 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.625516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.625584 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.625823 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.625826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.625940 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.626008 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.626012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.626703 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.626705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.627691 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.627699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.628092 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.628095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.628192 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.628537 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.628552 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.629596 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.630871 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.630873 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.630874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.631036 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.631224 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.631226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.631227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.631285 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.631590 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.631592 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.631657 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.631729 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.631820 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":24}
I20260620 07:07:12.632017 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.625450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.8.24.61:39679)
0620 07:07:12.625529 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.625533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.625554 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:07:12.625651 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.625654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.625686 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.625742 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.626127 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.626618 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.627781 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.628013 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.628087 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.629714 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.629717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.629720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.629740 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.630790 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.631106 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.631108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.631110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.631169 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.631340 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.631342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.631343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.631524 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.631592 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.631768 (+ 176us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.631865 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":35}
I20260620 07:07:12.634389 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Bootstrap starting.
I20260620 07:07:12.636538 14926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:12.637166 14926 log.cc:824] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:12.638892 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: No bootstrap required, opened a new log
I20260620 07:07:12.641705 14926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } }
I20260620 07:07:12.641884 14926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:12.641927 14926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 995e53db55434dc9938a2720cc84e872, State: Initialized, Role: FOLLOWER
I20260620 07:07:12.642305 14926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } }
I20260620 07:07:12.642726 14932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } } }
I20260620 07:07:12.642832 14932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:12.643371 14926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 07:07:12.647296 14943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:12.647358 14943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:12.649608 14858 master_runner.cc:428] Detected that this master 995e53db55434dc9938a2720cc84e872 is joining an existing cluster
I20260620 07:07:12.649652 14858 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.24.61:39679
I20260620 07:07:12.658766 14949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.651361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51436)
0620 07:07:12.651656 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.651670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.651752 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.651965 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.651968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.652171 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.652277 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.652282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.653161 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.653163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.653865 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.653873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.654447 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.654450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.654535 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.655060 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.655072 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.655485 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.657225 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.657228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.657230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.657501 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.657786 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.657789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.657791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.657887 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.658281 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.658284 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.658367 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.658471 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.658588 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":16,"thread_start_us":122,"threads_started":1}
I20260620 07:07:12.658812 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.651423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.8.24.62:42557)
0620 07:07:12.651757 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.651761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.651774 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:12.651815 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.651818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.651861 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.651925 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.652402 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.653056 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.653978 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.654320 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.654449 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.655611 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.655615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.655617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.655641 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.657126 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.657585 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.657589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.657591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.657676 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.657979 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.657983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.657984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.658199 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.658288 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.658553 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.658664 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260620 07:07:12.659251 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.652085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.8.24.61:39679)
0620 07:07:12.652189 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.652193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.652204 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:12.652232 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.652235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.652270 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.652344 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.652789 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.653541 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.654419 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.654703 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.654788 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.656166 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.656170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.656172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.656195 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.657713 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.658097 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.658100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.658101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.658164 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.658410 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.658413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.658415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.658631 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.658711 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.659005 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.659125 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":44}
I20260620 07:07:12.659678 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.651375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53404)
0620 07:07:12.651752 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.651765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.651847 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.652374 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.652376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.652560 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.652684 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.652691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.653646 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.653649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.654312 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.654320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.654790 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.654793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.654898 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.655642 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.655656 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.656028 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.657801 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.657804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.657805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.658007 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.658251 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.658253 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.658255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.658328 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.658779 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.658781 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.658855 (+ 74us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.659393 (+ 538us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.659525 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":46,"spinlock_wait_cycles":1920,"thread_start_us":110,"threads_started":1}
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260620 07:07:12.671201 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.663086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.8.24.61:39679)
0620 07:07:12.663175 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.663178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.663524 (+ 346us) server_negotiation.cc:407] Connection header received
0620 07:07:12.663732 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.663735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.663780 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.663838 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.664598 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.665033 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.666172 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.666361 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.666429 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.668612 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.668615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.668618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.668639 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.669777 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.670217 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.670219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.670220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.670279 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.670515 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.670517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.670517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.670653 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.670717 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.671035 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.671105 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 07:07:12.671339 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.663085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53408)
0620 07:07:12.663378 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.663460 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.663611 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.663881 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.663887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.664354 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.664493 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.664500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.665118 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.665121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.666076 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.666086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.666472 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.666476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.666579 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.666994 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.667007 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.668484 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.669865 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.669872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.669878 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.670137 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.670343 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.670346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.670348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.670439 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.670733 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.670739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.670861 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.671006 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.671177 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260620 07:07:12.672523 14394 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.24.61:39679: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "995e53db55434dc9938a2720cc84e872"
member_type: NON_VOTER
last_known_addr {
host: "127.8.24.61"
port: 39679
}
attrs {
promote: true
}
}
dest_uuid: "f277a978387143d89c960def5b77cf24"
cas_config_opid_index: -1
I20260620 07:07:12.673079 14394 consensus_queue.cc:237] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } }
I20260620 07:07:12.674484 14953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 995e53db55434dc9938a2720cc84e872 (127.8.24.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } }
I20260620 07:07:12.675251 14900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:12.676116 14953 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.24.61:39679
I20260620 07:07:12.676565 14858 master.cc:537] Master@127.8.24.61:39679 shutting down...
I20260620 07:07:12.676611 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 07:07:12.676625 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } }
I20260620 07:07:12.676710 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:12.677404 14952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 07:07:12.679139 14380 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260620 07:07:12.681895 14858 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:12.682215 14858 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:12.682371 14858 tablet_replica.cc:330] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: stopping tablet replica
I20260620 07:07:12.687005 14858 master.cc:552] Master@127.8.24.61:39679 shutdown complete.
I20260620 07:07:12.687096 14858 master_runner.cc:305] Clearing existing system tablet
I20260620 07:07:12.687382 14858 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.687505 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
WARNING: no policy specified for oryx/127.8.24.60@KRBTEST.COM; defaulting to no policy
I20260620 07:07:12.688117 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.687826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53416)
0620 07:07:12.687979 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.688039 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260620 07:07:12.688265 14380 consensus_peers.cc:487] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 -> Peer 995e53db55434dc9938a2720cc84e872 (127.8.24.61:39679): Couldn't send request to peer 995e53db55434dc9938a2720cc84e872. Status: Network error: Client connection negotiation failed: client connection to 127.8.24.61:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 07:07:12.688731 14858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.60@KRBTEST.COM" created.
I20260620 07:07:12.690075 14958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.690245 14858 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.690361 14858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 07:07:12.690411 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "995e53db55434dc9938a2720cc84e872"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "33c434dad4e8fa8109284f077591ac86"
server_key_iv: "c6edecabe0f2ba8ec3f1dea2f8b11955"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.692148 14858 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:12.694478 14858 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 07:07:12.694567 14858 log.cc:1196] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 07:07:12.694872 14858 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Deleting consensus metadata
I20260620 07:07:12.695034 14858 master_runner.cc:315] Copying system tablet from 127.8.24.62:42557
I20260620 07:07:12.696223 14858 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: tablet copy: Beginning tablet copy session from remote peer at address 127.8.24.62:42557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.60.keytab.
Entry for principal oryx/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.60.keytab.
I20260620 07:07:12.704221 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.696718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51448)
0620 07:07:12.697061 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.697077 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.697195 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.697463 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.697466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.697661 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.698016 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.698025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.699061 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.699064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.699737 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.699746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.700284 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.700287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.700358 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.700833 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.700843 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.701179 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.702823 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.702826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.702827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.703007 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.703256 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.703258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.703260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.703331 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.703750 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.703753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.703848 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.703950 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.704076 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":74,"threads_started":1}
I20260620 07:07:12.704546 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:41943
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.60.keytab
--principal=oryx/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:07:12.706261 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.696807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.8.24.62:42557)
0620 07:07:12.696920 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.696924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.697165 (+ 241us) server_negotiation.cc:407] Connection header received
0620 07:07:12.697291 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.697294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.697346 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.697433 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.698149 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.698951 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.699844 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.700184 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.700282 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.701300 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.701304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.701307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.701334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.702714 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.703097 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.703101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.703103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.703183 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.703408 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.703411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.703413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.703649 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.703812 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.705998 (+ 2186us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.706114 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:12.706789 14414 tablet_copy_service.cc:140] P f277a978387143d89c960def5b77cf24: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 995e53db55434dc9938a2720cc84e872 ({username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:51448)
I20260620 07:07:12.706955 14414 tablet_copy_service.cc:161] P f277a978387143d89c960def5b77cf24: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 995e53db55434dc9938a2720cc84e872 at {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:51448: session id = 995e53db55434dc9938a2720cc84e872-00000000000000000000000000000000
I20260620 07:07:12.708669 14414 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 07:07:12.710274 14858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 07:07:12.712597 14858 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: tablet copy: Starting download of 0 data blocks...
I20260620 07:07:12.712780 14858 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: tablet copy: Starting download of 1 WAL segments...
I20260620 07:07:12.714438 14858 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 07:07:12.716063 14414 tablet_copy_service.cc:342] P f277a978387143d89c960def5b77cf24: Request end of tablet copy session 995e53db55434dc9938a2720cc84e872-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:51448
I20260620 07:07:12.716172 14414 tablet_copy_service.cc:433] P f277a978387143d89c960def5b77cf24: ending tablet copy session 995e53db55434dc9938a2720cc84e872-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 995e53db55434dc9938a2720cc84e872
I20260620 07:07:12.723676 14858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:12.725137 14967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.725239 14968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.725373 14970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:12.726099 14858 server_base.cc:1034] running on GCE node
I20260620 07:07:12.726238 14858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:12.726517 14858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:12.727705 14858 hybrid_clock.cc:648] HybridClock initialized: now 1781939232727693 us; error 28 us; skew 500 ppm
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:12.729209 14858 init.cc:377] Logged in from keytab as oryx/127.8.24.61@KRBTEST.COM (short username oryx)
I20260620 07:07:12.729632 14858 webserver.cc:460] Webserver started at http://127.8.24.61:35223/ using document root <none> and password file <none>
I20260620 07:07:12.729807 14858 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.729846 14858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:12.731052 14858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:12.731971 14977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.732221 14858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 07:07:12.732295 14858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "995e53db55434dc9938a2720cc84e872"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "33c434dad4e8fa8109284f077591ac86"
server_key_iv: "c6edecabe0f2ba8ec3f1dea2f8b11955"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.732369 14858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.772493 14858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:12.774677 14858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:12.779223 14858 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:39679
I20260620 07:07:12.779788 14858 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
I20260620 07:07:12.780159 15029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:39679 every 8 connection(s)
I20260620 07:07:12.781472 15030 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:07:12.782394 15030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Bootstrap starting.
I20260620 07:07:12.791471 15030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:07:12.791718 15030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Bootstrap complete.
I20260620 07:07:12.792212 15030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } }
I20260620 07:07:12.792342 15030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 995e53db55434dc9938a2720cc84e872, State: Initialized, Role: LEARNER
I20260620 07:07:12.792488 15030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } }
I20260620 07:07:12.792780 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } }
I20260620 07:07:12.792901 15030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:12.792944 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: LEARNER
I20260620 07:07:12.795868 15044 catalog_manager.cc:1259] Loaded cluster ID: 45ad0780c6dc4b90b9329ca783f0f075
I20260620 07:07:12.795928 15044 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: loading cluster ID for follower catalog manager: success
I20260620 07:07:12.797952 15044 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: acquiring CA information for follower catalog manager: success
I20260620 07:07:12.799379 15044 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 07:07:12.829151 14966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:12.829380 14966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:12.829429 14966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:12.833343 14966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:12.833385 14966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:12.833407 14966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:12.833427 14966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:12.833452 14966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:12.838423 14966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.60.keytab
--principal=oryx/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:41943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:12.839287 14966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:12.840217 14966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:12.847716 15052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.847683 15049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:12.847693 15050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:12.848040 14966 server_base.cc:1034] running on GCE node
I20260620 07:07:12.848551 14966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:12.849530 14966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:12.850682 14966 hybrid_clock.cc:648] HybridClock initialized: now 1781939232850666 us; error 33 us; skew 500 ppm
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:12.853207 14966 init.cc:377] Logged in from keytab as oryx/127.8.24.60@KRBTEST.COM (short username oryx)
I20260620 07:07:12.853974 14966 webserver.cc:460] Webserver started at http://127.8.24.60:46149/ using document root <none> and password file <none>
I20260620 07:07:12.854486 14966 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.854548 14966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:12.854720 14966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:12.856659 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/instance:
uuid: "263c09629cb347c389e73fea9441e609"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "be751e5e19f9ea74ddf25954046c4871"
server_key_iv: "57bd75e552753b5bc3ced55cf31a7603"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.857163 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal/instance:
uuid: "263c09629cb347c389e73fea9441e609"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "be751e5e19f9ea74ddf25954046c4871"
server_key_iv: "57bd75e552753b5bc3ced55cf31a7603"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.860689 14966 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:12.862805 15059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:12.864015 14966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:12.864114 14966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "263c09629cb347c389e73fea9441e609"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "be751e5e19f9ea74ddf25954046c4871"
server_key_iv: "57bd75e552753b5bc3ced55cf31a7603"
server_key_version: "encryptionkey@0"
I20260620 07:07:12.864189 14966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:12.878756 14966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:12.881212 14966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:12.881343 14966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:12.887259 14966 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:41943
I20260620 07:07:12.887589 15111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:41943 every 8 connection(s)
I20260620 07:07:12.888212 14966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
I20260620 07:07:12.890347 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 14966
I20260620 07:07:12.890779 15112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:12.895712 15112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:12.903776 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.891607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.8.24.60:41943)
0620 07:07:12.891952 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.891966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.891999 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:07:12.892612 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.892627 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.892998 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.893263 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.894155 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.895188 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.896328 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.896766 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.896879 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.899331 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.899335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.899343 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.899367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.901098 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.901470 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.901476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.901481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.901580 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.901827 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.901829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.901830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.902189 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.902335 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.902608 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.902830 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":109,"threads_started":1}
I20260620 07:07:12.905375 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.897438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.8.24.62:42557)
0620 07:07:12.897555 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.897560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.897752 (+ 192us) server_negotiation.cc:407] Connection header received
0620 07:07:12.897871 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.897874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.897926 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.898014 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.898585 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.899128 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.900125 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.900339 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.900415 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.902411 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.902415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.902418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.902440 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.903861 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.904275 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.904278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.904280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.904363 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.904586 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.904588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.904590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.904757 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.904838 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.905138 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.905233 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:12.905397 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.897328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51452)
0620 07:07:12.897637 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.897682 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.897785 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.898047 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.898052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.898350 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.898456 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.898461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.899241 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.899243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.900010 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.900019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.900443 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.900447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.900551 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.900957 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.900972 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.902278 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.903972 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.903980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.903985 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.904185 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.904444 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.904446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.904447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.904509 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.904850 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.904856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.904961 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.905071 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.905230 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260620 07:07:12.906452 15112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260620 07:07:12.913066 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.906795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53418)
0620 07:07:12.906889 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.906901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.906956 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.907279 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.907281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.907385 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.907489 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.907495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.908306 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.908309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.908799 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.908806 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.909159 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.909161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.909220 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.909552 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.909562 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.910542 (+ 980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.911872 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.911875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.911877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.912041 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.912279 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.912281 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.912283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.912352 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.912665 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.912668 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.912745 (+ 77us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.912822 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.912925 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 07:07:12.913136 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.906890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.8.24.61:39679)
0620 07:07:12.907113 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.907116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.907127 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:12.907159 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.907161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.907197 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.907264 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.907563 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.908209 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.908880 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.909093 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.909157 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.910669 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.910671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.910674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.910695 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.911782 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.912130 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.912133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.912134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.912202 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.912425 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.912428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.912430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.912583 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.912661 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.912880 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.912982 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260620 07:07:12.913722 15112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939232, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260620 07:07:12.920017 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.914051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33696)
0620 07:07:12.914134 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.914144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.914194 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.914377 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.914379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.914483 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.914540 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.914544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.915461 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.915465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.915929 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.915936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.916304 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.916307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.916378 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.916717 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.916727 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.917731 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.918954 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.918957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.918958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.919142 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.919348 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.919350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.919351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.919411 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.919703 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.919705 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.919768 (+ 63us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.919835 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.919919 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":13}
I20260620 07:07:12.920090 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.914146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.8.24.60:41943)
0620 07:07:12.914233 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.914236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.914246 (+ 10us) server_negotiation.cc:407] Connection header received
0620 07:07:12.914269 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.914271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.914298 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.914353 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.914613 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.915362 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.916013 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.916235 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.916300 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.917825 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.917828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.917829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.917846 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.918862 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.919216 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.919218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.919219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.919287 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.919506 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.919508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.919509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.919640 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.919704 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.919864 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.919964 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":34}
I20260620 07:07:12.922722 15112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Bootstrap starting.
I20260620 07:07:12.924751 15112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:12.925367 15112 log.cc:824] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:12.926896 15112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: No bootstrap required, opened a new log
I20260620 07:07:12.929615 15112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } }
I20260620 07:07:12.929785 15112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:12.929831 15112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263c09629cb347c389e73fea9441e609, State: Initialized, Role: FOLLOWER
I20260620 07:07:12.930208 15112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } }
I20260620 07:07:12.930583 15118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } } }
I20260620 07:07:12.930693 15118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:12.931197 15112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 07:07:12.935364 15129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:12.935420 15129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:12.938203 14966 master_runner.cc:428] Detected that this master 263c09629cb347c389e73fea9441e609 is joining an existing cluster
I20260620 07:07:12.938256 14966 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.24.60:41943
I20260620 07:07:12.947976 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33698)
0620 07:07:12.941146 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.941157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.941244 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.941494 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.941497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.941736 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.941926 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.941931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.942865 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.942867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.943715 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.943723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944283 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.944365 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.944791 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.944805 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.945120 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.946762 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.946764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.946766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.946935 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.947183 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.947185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.947186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.947249 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.947615 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.947617 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.947686 (+ 69us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.947755 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.947841 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":57,"spinlock_wait_cycles":3328,"thread_start_us":64,"threads_started":1}
I20260620 07:07:12.948112 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.8.24.60:41943)
0620 07:07:12.941002 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.941005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.941206 (+ 201us) server_negotiation.cc:407] Connection header received
0620 07:07:12.941318 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.941320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.941349 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.941386 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.942028 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.942759 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.943819 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944106 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944166 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.945237 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.945241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.945244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.945268 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.946672 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.947013 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.947016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.947018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.947106 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.947315 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.947318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.947319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.947537 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.947621 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.947823 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.947927 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:07:12.956717 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14456
I20260620 07:07:12.961766 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:37015
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:42557
--webserver_port=40461
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:07:12.968122 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53420)
0620 07:07:12.940726 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.940739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.940829 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.941164 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.941167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.941389 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.941510 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.941518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.942581 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.942584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.943530 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.943542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944238 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.944327 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.963985 (+ 19658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.964004 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.964498 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.966350 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.966353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.966355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.966579 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.966885 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.966888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.966890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.966995 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.967523 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.967526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.967659 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.967760 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.967909 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":86,"threads_started":1}
I20260620 07:07:12.968869 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.8.24.61:39679)
0620 07:07:12.940644 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.940648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.940791 (+ 143us) server_negotiation.cc:407] Connection header received
0620 07:07:12.940918 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.940921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.940968 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.941043 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.941642 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.942448 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.943662 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944000 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944072 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.964668 (+ 20596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.964672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.964675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.964709 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.966222 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.966695 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.966698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.966700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.966784 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.967134 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.967137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.967139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.967367 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.967478 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.968587 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.968719 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:07:12.968879 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51466)
0620 07:07:12.941002 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.941014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.941130 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.941450 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.941453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.941730 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.941924 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.941931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.943032 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.943035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.943854 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.943860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944453 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.944794 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.964186 (+ 19392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.964202 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.964657 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.966948 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.966951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.966953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.967250 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.967560 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.967563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.967565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.967643 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.968094 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.968097 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.968469 (+ 372us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.968574 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.968702 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":45,"spinlock_wait_cycles":9472,"thread_start_us":68,"threads_started":1}
I20260620 07:07:12.969282 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.940430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.8.24.62:42557)
0620 07:07:12.940535 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.940539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.941074 (+ 535us) server_negotiation.cc:407] Connection header received
0620 07:07:12.941234 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.941237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.941282 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.941328 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.942042 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.942811 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.943955 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.944262 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.944325 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.964835 (+ 20510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.964840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.964843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.964873 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.966819 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.967354 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.967357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.967359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.967456 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.967731 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.967734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.967736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.967929 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.968007 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.968998 (+ 991us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.969135 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 07:07:12 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939231, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260620 07:07:12.983952 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.973889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33700)
0620 07:07:12.974101 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.974114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.974181 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.976003 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.976006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.976189 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:12.976285 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.976292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.977271 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.977274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.978124 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.978133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.978652 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.978656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.978752 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.979195 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:12.979208 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:12.980658 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:12.982390 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.982394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.982396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.982633 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.982884 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:12.982887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:12.982889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:12.982970 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:12.983324 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:12.983327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:12.983416 (+ 89us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.983666 (+ 250us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.983798 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260620 07:07:12.983968 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.974309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.8.24.60:41943)
0620 07:07:12.975751 (+ 1442us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.975756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.975768 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:12.975831 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.975834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.975880 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.975951 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:12.976385 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.977170 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.978225 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.978560 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.978649 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.980798 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:12.980801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:12.980804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:12.980827 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:12.982276 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.982718 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.982722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.982724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.982798 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:12.983045 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:12.983047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:12.983049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:12.983244 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:12.983328 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.983733 (+ 405us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.983833 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1370}
I20260620 07:07:12.984998 14394 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.24.60:41943: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "263c09629cb347c389e73fea9441e609"
member_type: NON_VOTER
last_known_addr {
host: "127.8.24.60"
port: 41943
}
attrs {
promote: true
}
}
dest_uuid: "f277a978387143d89c960def5b77cf24"
cas_config_opid_index: 5
I20260620 07:07:12.985579 14394 consensus_queue.cc:237] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } }
I20260620 07:07:12.986598 14952 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 263c09629cb347c389e73fea9441e609 (127.8.24.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:12.987346 15086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:12.987942 14952 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.24.60:41943
I20260620 07:07:12.988096 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:12.988179 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:12.988636 14966 master.cc:537] Master@127.8.24.60:41943 shutting down...
I20260620 07:07:12.988780 15086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 07:07:12.989456 14953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 07:07:12.991247 14380 consensus_peers.cc:487] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 -> Peer 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943): Couldn't send request to peer 263c09629cb347c389e73fea9441e609. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 07:07:12.993456 14966 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:12.993709 14966 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:12.993809 14966 tablet_replica.cc:330] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: stopping tablet replica
I20260620 07:07:12.996104 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.986630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53430)
0620 07:07:12.991573 (+ 4943us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:12.991594 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:12.991667 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:12.992133 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:12.992137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:12.992152 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:12.992227 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.992233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.993250 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.993253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.994673 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:12.994683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.995721 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.995725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:12.995810 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.995852 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 07:07:12.995919 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:07:12.995982 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4884}
I20260620 07:07:12.997159 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:12.986827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.8.24.61:39679)
0620 07:07:12.991493 (+ 4666us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:12.991497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:12.991819 (+ 322us) server_negotiation.cc:407] Connection header received
0620 07:07:12.991875 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:12.991878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:12.991934 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:12.992011 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:12.992337 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.993132 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.994802 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:12.995409 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:12.995529 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:12.995665 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:12.996315 (+ 650us) server_negotiation.cc:299] Negotiation successful
0620 07:07:12.997037 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4555}
I20260620 07:07:12.997874 14996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 LEARNER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 07:07:12.998380 14953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 07:07:12.998667 14966 master.cc:552] Master@127.8.24.60:41943 shutdown complete.
I20260620 07:07:12.998710 14966 master_runner.cc:305] Clearing existing system tablet
I20260620 07:07:12.998919 14966 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:12.998957 14966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:13.000304 14996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 263c09629cb347c389e73fea9441e609 (127.8.24.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.002367 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f277a978387143d89c960def5b77cf24. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.002466 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: LEARNER
I20260620 07:07:13.002544 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: NON_VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: true } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.002602 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: LEARNER
I20260620 07:07:13.003222 14952 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: attempting to promote NON_VOTER 995e53db55434dc9938a2720cc84e872 to VOTER
I20260620 07:07:13.003509 14966 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260620 07:07:13.003682 14952 consensus_queue.cc:237] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } }
I20260620 07:07:13.004802 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.004496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33708)
0620 07:07:13.004695 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.004742 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:41943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260620 07:07:13.005165 14996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 LEARNER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 07:07:13.005676 15142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 07:07:13.006372 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.005945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33724)
0620 07:07:13.006262 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.006311 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:41943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253}
W20260620 07:07:13.007015 14380 consensus_peers.cc:487] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 -> Peer 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943): Couldn't send request to peer 263c09629cb347c389e73fea9441e609. Status: Network error: Client connection negotiation failed: client connection to 127.8.24.60:41943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 07:07:13.007344 15143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.007635 14953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 995e53db55434dc9938a2720cc84e872 (127.8.24.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.008116 14996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 995e53db55434dc9938a2720cc84e872 (127.8.24.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.009411 14953 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.009511 14953 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.009625 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.009701 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.007537 14966 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:13.010000 14966 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260620 07:07:13.010044 14966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "263c09629cb347c389e73fea9441e609"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "be751e5e19f9ea74ddf25954046c4871"
server_key_iv: "57bd75e552753b5bc3ced55cf31a7603"
server_key_version: "encryptionkey@0"
I20260620 07:07:13.011590 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.011767 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:13.011946 14966 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:13.014024 14966 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 07:07:13.014108 14966 log.cc:1196] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 07:07:13.014323 14966 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Deleting consensus metadata
I20260620 07:07:13.014480 14966 master_runner.cc:315] Copying system tablet from 127.8.24.62:42557
I20260620 07:07:13.015532 14966 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: tablet copy: Beginning tablet copy session from remote peer at address 127.8.24.62:42557
I20260620 07:07:13.031334 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.016081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.8.24.62:42557)
0620 07:07:13.016187 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.016191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.023735 (+ 7544us) server_negotiation.cc:407] Connection header received
0620 07:07:13.023884 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.023887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.023939 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.024015 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.024486 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.025286 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.026418 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.026772 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.026868 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.028087 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.028092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.028094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.028122 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.029676 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.030141 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.030144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.030146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.030231 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.030475 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.030478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.030480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.030677 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.030767 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.031060 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.031182 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:13.031365 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.016013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51472)
0620 07:07:13.023624 (+ 7611us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.023641 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.023769 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.024046 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.024048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.024272 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.024380 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.024387 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.025416 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.025420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.026295 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.026306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.026897 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.026900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.026998 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.027538 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.027552 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.027942 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.029791 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.029795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.029798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.030043 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.030311 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.030314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.030316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.030399 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.030769 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.030773 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.030869 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.030998 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.031174 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7531,"thread_start_us":97,"threads_started":1}
I20260620 07:07:13.032171 14414 tablet_copy_service.cc:140] P f277a978387143d89c960def5b77cf24: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 263c09629cb347c389e73fea9441e609 ({username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:51472)
I20260620 07:07:13.032262 14414 tablet_copy_service.cc:161] P f277a978387143d89c960def5b77cf24: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 263c09629cb347c389e73fea9441e609 at {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:51472: session id = 263c09629cb347c389e73fea9441e609-00000000000000000000000000000000
I20260620 07:07:13.033555 14414 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 07:07:13.034813 14966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 07:07:13.037258 14966 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: tablet copy: Starting download of 0 data blocks...
I20260620 07:07:13.037462 14966 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: tablet copy: Starting download of 1 WAL segments...
I20260620 07:07:13.038786 14966 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 07:07:13.040591 14414 tablet_copy_service.cc:342] P f277a978387143d89c960def5b77cf24: Request end of tablet copy session 263c09629cb347c389e73fea9441e609-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.24.60@KRBTEST.COM'} at 127.0.0.1:51472
I20260620 07:07:13.040681 14414 tablet_copy_service.cc:433] P f277a978387143d89c960def5b77cf24: ending tablet copy session 263c09629cb347c389e73fea9441e609-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 263c09629cb347c389e73fea9441e609
I20260620 07:07:13.048142 14966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:13.049659 15151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:13.052568 14966 server_base.cc:1034] running on GCE node
W20260620 07:07:13.052711 15154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.055821 15152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:13.056083 14966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:13.056444 14966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:13.057595 14966 hybrid_clock.cc:648] HybridClock initialized: now 1781939233057572 us; error 41 us; skew 500 ppm
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:13.059288 14966 init.cc:377] Logged in from keytab as oryx/127.8.24.60@KRBTEST.COM (short username oryx)
I20260620 07:07:13.059813 14966 webserver.cc:460] Webserver started at http://127.8.24.60:36763/ using document root <none> and password file <none>
I20260620 07:07:13.060003 14966 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:13.060053 14966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:13.061540 14966 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:13.062609 15161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.062980 14966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:13.063061 14966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "263c09629cb347c389e73fea9441e609"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "be751e5e19f9ea74ddf25954046c4871"
server_key_iv: "57bd75e552753b5bc3ced55cf31a7603"
server_key_version: "encryptionkey@0"
I20260620 07:07:13.063149 14966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:13.081444 14966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:13.083514 14966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:13.088275 14966 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:41943
I20260620 07:07:13.088900 14966 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
I20260620 07:07:13.089260 15213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:41943 every 8 connection(s)
I20260620 07:07:13.090628 15214 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:07:13.091696 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Bootstrap starting.
W20260620 07:07:13.095741 15138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:13.096062 15138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:13.096133 15138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:13.101716 15138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:13.101785 15138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:13.101872 15138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:13.102957 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:07:13.103205 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: Bootstrap complete.
I20260620 07:07:13.103744 15214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } }
I20260620 07:07:13.103876 15214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 263c09629cb347c389e73fea9441e609, State: Initialized, Role: LEARNER
I20260620 07:07:13.103974 15214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } }
I20260620 07:07:13.104259 15220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.104365 15220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: LEARNER
I20260620 07:07:13.104377 15214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:13.106913 15231 catalog_manager.cc:1259] Loaded cluster ID: 45ad0780c6dc4b90b9329ca783f0f075
I20260620 07:07:13.106963 15231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: loading cluster ID for follower catalog manager: success
I20260620 07:07:13.108471 15138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:37015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=40461
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:13.108557 15231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: acquiring CA information for follower catalog manager: success
I20260620 07:07:13.109470 15138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:13.109633 15231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:07:13.110424 15138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:13.119246 15236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.119278 15233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.119271 15234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:13.119408 15138 server_base.cc:1034] running on GCE node
I20260620 07:07:13.119940 15138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:13.120558 15138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:13.121711 15138 hybrid_clock.cc:648] HybridClock initialized: now 1781939233121693 us; error 29 us; skew 500 ppm
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:13.124334 15138 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260620 07:07:13.125166 15138 webserver.cc:460] Webserver started at http://127.8.24.1:40461/ using document root <none> and password file <none>
I20260620 07:07:13.125686 15138 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:13.125764 15138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:13.129489 15138 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260620 07:07:13.131387 15243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.132589 15138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:07:13.132696 15138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "45c3a263a49b440990db588277f98150"
format_stamp: "Formatted at 2026-06-20 07:07:11 on dist-test-slave-wjw3"
server_key: "733fbcc19badd65ce69d11e6c20c3af3"
server_key_iv: "0927afe0d40e944da060a28fe91c93d2"
server_key_version: "encryptionkey@0"
I20260620 07:07:13.133234 15138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:13.168426 15138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:13.171726 15138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:13.171901 15138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:13.172485 15138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:13.173322 15138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:13.173369 15138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.173420 15138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:13.173445 15138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.182102 15138 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37015
I20260620 07:07:13.182149 15356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37015 every 8 connection(s)
I20260620 07:07:13.183055 15138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:13.191730 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 15138
I20260620 07:07:13.191918 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14590
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:13.197365 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.184754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46235 (local address 127.8.24.62:42557)
0620 07:07:13.184912 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.184916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.186410 (+ 1494us) server_negotiation.cc:407] Connection header received
0620 07:07:13.186835 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.186839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.186894 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.186944 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.188300 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.188823 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190166 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190418 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190512 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.193942 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.193946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.193949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.193978 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.195270 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.195919 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.195922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.195924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.196004 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.196364 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.196367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.196369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.196570 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.196658 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.197110 (+ 452us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.197218 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 07:07:13.198392 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.185184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:46235)
0620 07:07:13.186322 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.186335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.186632 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.187110 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.187115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.187661 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.188163 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.188176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.188951 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.188954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.190035 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.190044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190626 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.190820 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.191621 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.191667 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.193820 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.195373 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.195378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.195392 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.195802 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.196095 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.196097 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.196099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.196261 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.196656 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.196661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.196860 (+ 199us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.197046 (+ 186us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.197455 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":191,"spinlock_wait_cycles":11648,"thread_start_us":99,"threads_started":1}
I20260620 07:07:13.198392 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.186093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.8.24.1:59581)
0620 07:07:13.186356 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.186367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.186633 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.187110 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.187115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.187685 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.188164 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.188178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190111 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.190789 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.190797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.191230 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.191234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.191341 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.191748 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.191757 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.193436 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.195428 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.195452 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.195454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.195802 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.196076 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.196079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.196081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.196256 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.196988 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.196991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.197080 (+ 89us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.197158 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.197398 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":73,"spinlock_wait_cycles":2816,"thread_start_us":69,"threads_started":1}
I20260620 07:07:13.198997 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.186182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:59581 (local address 127.8.24.60:41943)
0620 07:07:13.186570 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.186575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.186590 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:07:13.186899 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.186902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.186950 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.187236 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.188953 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190008 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190884 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.191124 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.191213 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.193673 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.193677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.193679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.193704 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.195311 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.195900 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.195904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.195905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.195984 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.196707 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.196710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.196712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.196901 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.196977 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.198747 (+ 1770us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.198869 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":83,"threads_started":1}
I20260620 07:07:13.199726 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:45503
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:42557
--webserver_port=39387
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:07:13.199805 15358 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:13.199985 15358 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.200434 15358 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
I20260620 07:07:13.201057 15357 heartbeater.cc:347] Connected to a master server at 127.8.24.60:41943
I20260620 07:07:13.201143 15357 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.201283 15357 heartbeater.cc:511] Master 127.8.24.60:41943 requested a full tablet report, sending...
I20260620 07:07:13.201905 14394 ts_manager.cc:194] Re-registered known tserver with Master: 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
I20260620 07:07:13.202277 15178 ts_manager.cc:194] Registered new tserver with Master: 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
I20260620 07:07:13.202872 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:46235
I20260620 07:07:13.204574 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.184926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39357 (local address 127.8.24.61:39679)
0620 07:07:13.185037 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.185049 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.185785 (+ 736us) server_negotiation.cc:407] Connection header received
0620 07:07:13.186815 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.186818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.186872 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.186938 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.188313 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.188835 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.189990 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190272 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190369 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.201252 (+ 10883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.201256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.201258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.201285 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.202813 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.203257 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.203260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.203262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.203350 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.203680 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.203683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.203685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.203878 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.203946 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.204313 (+ 367us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.204427 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:07:13.204782 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.185184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.8.24.1:39357)
0620 07:07:13.185631 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.185680 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.186631 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.187108 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.187114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.187942 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.188164 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.188178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.188957 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.188960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.189889 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.189898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.190408 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.190411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.199417 (+ 9006us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.199827 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.199841 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.201113 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.202932 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.202935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.202937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.203156 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.203481 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.203484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.203486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.203579 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.204013 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.204016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.204130 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.204224 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.204356 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":74,"thread_start_us":95,"threads_started":1}
I20260620 07:07:13.205466 15359 heartbeater.cc:347] Connected to a master server at 127.8.24.61:39679
I20260620 07:07:13.205538 15359 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.205695 15359 heartbeater.cc:511] Master 127.8.24.61:39679 requested a full tablet report, sending...
I20260620 07:07:13.206573 14986 ts_manager.cc:194] Registered new tserver with Master: 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
W20260620 07:07:13.305121 15365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:13.305367 15365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:13.305423 15365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:13.309401 15365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:13.309450 15365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:13.309525 15365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:13.314452 15365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:45503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=39387
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:13.315351 15365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:13.316295 15365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:13.324807 15371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.324800 15373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.324803 15370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:13.324833 15365 server_base.cc:1034] running on GCE node
I20260620 07:07:13.325343 15365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:13.325882 15365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:13.327020 15365 hybrid_clock.cc:648] HybridClock initialized: now 1781939233326995 us; error 39 us; skew 500 ppm
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:13.329419 15365 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260620 07:07:13.330186 15365 webserver.cc:460] Webserver started at http://127.8.24.2:39387/ using document root <none> and password file <none>
I20260620 07:07:13.330679 15365 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:13.330747 15365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:13.334246 15365 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:13.336069 15380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.337172 15365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:13.337257 15365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "5e8c643e69c246ed8a84dd255c133453"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "710cc24431a283b52af5f2c4cc49ed19"
server_key_iv: "a3c81f4c53147a611025e00116908368"
server_key_version: "encryptionkey@0"
I20260620 07:07:13.337739 15365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:13.354086 15365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:13.356632 15365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:13.356760 15365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:13.357239 15365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:13.357964 15365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:13.358006 15365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.358049 15365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:13.358076 15365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.366410 15365 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:45503
I20260620 07:07:13.366432 15493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:45503 every 8 connection(s)
I20260620 07:07:13.367306 15365 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:13.375327 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 15365
I20260620 07:07:13.375523 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14724
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
I20260620 07:07:13.381747 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.370598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37089 (local address 127.8.24.61:39679)
0620 07:07:13.370708 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.370712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.370725 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:13.371122 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.371125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.371174 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.371222 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.372619 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373412 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374419 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.374637 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374719 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.377980 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.377984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.377986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.378014 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.379505 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380092 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.380096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.380098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.380188 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380562 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.380566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.380568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.380793 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.380857 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.381482 (+ 625us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.381597 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 07:07:13.382236 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.369104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:41857 (local address 127.8.24.62:42557)
0620 07:07:13.369212 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.369215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.370108 (+ 893us) server_negotiation.cc:407] Connection header received
0620 07:07:13.371194 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.371197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.371247 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.371311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.372620 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373145 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374589 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.374873 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374975 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.377774 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.377778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.377781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.377809 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.380272 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380703 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.380706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.380707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.380788 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.381057 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.381060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.381062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.381248 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.381329 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.382047 (+ 718us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.382135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:07:13.382534 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.368971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:38173 (local address 127.8.24.60:41943)
0620 07:07:13.369097 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.369102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.370119 (+ 1017us) server_negotiation.cc:407] Connection header received
0620 07:07:13.371107 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.371111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.371168 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.371240 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.372607 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373412 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374726 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.375035 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.375129 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.379252 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.379256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.379258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.379287 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.380882 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.381304 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.381308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.381310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.381393 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.381675 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.381678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.381680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.381870 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.381933 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.382273 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.382389 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 07:07:13.383289 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.369490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:41857)
0620 07:07:13.369964 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.370015 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.370919 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.371411 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.371419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.371921 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.372472 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.372481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373267 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.373270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.374457 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.374466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.374978 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.375120 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.376138 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.376153 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.377588 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.380376 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.380380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.380607 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.380882 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.380888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.380965 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.381366 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.381369 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.381447 (+ 78us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.381544 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.382449 (+ 905us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":155,"spinlock_wait_cycles":49792,"thread_start_us":113,"threads_started":1}
I20260620 07:07:13.383574 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.369490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.8.24.2:38173)
0620 07:07:13.369965 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.370015 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.370917 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.371412 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.371418 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.371913 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.372464 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.372477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373815 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.373818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.374624 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.374632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.375523 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.375527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.376860 (+ 1333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.377769 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.377783 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.379107 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.380992 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.380997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.381207 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.381495 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.381497 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.381499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.381582 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.382001 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.382004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.382110 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.382704 (+ 594us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.382831 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":74,"spinlock_wait_cycles":43264,"thread_start_us":100,"threads_started":1}
I20260620 07:07:13.383806 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.370429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.8.24.2:37089)
0620 07:07:13.370663 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.370675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.370927 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.371411 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.371419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.371921 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.372475 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.372481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.373515 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.373518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.374307 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.374316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.374869 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.374873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.375110 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.376272 (+ 1162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.376288 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.377821 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.379628 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.379634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.379648 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.379965 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.380283 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.380286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.380288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.380454 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.380928 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.380934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.381233 (+ 299us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.381890 (+ 657us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.383656 (+ 1766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1739,"spinlock_wait_cycles":36864,"thread_start_us":79,"threads_started":1}
I20260620 07:07:13.384478 15494 heartbeater.cc:347] Connected to a master server at 127.8.24.60:41943
I20260620 07:07:13.384486 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:36375
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:42557
--webserver_port=36791
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:07:13.384503 15495 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:13.384716 15494 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.384716 15495 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.385201 15494 heartbeater.cc:511] Master 127.8.24.60:41943 requested a full tablet report, sending...
I20260620 07:07:13.385201 15495 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
I20260620 07:07:13.386245 15178 ts_manager.cc:194] Registered new tserver with Master: 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503)
I20260620 07:07:13.386294 14394 ts_manager.cc:194] Re-registered known tserver with Master: 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503)
I20260620 07:07:13.386869 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:41857
I20260620 07:07:13.387084 15496 heartbeater.cc:347] Connected to a master server at 127.8.24.61:39679
I20260620 07:07:13.387164 15496 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.387308 15496 heartbeater.cc:511] Master 127.8.24.61:39679 requested a full tablet report, sending...
I20260620 07:07:13.387971 14986 ts_manager.cc:194] Registered new tserver with Master: 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503)
I20260620 07:07:13.458695 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.455626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.8.24.60:41943)
0620 07:07:13.455739 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.455744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.455760 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:07:13.455879 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.455883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.455934 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.456010 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.456256 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.456765 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.457744 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.458166 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.458261 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.458416 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.458492 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.458558 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:07:13.458695 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.455533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33734)
0620 07:07:13.455660 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.455676 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.455779 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.456038 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.456041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.456063 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.456158 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.456163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.456878 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.456882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.457625 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.457635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.458286 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.458290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.458378 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.458400 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.458478 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.458566 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 07:07:13.459398 15188 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260620 07:07:13.459863 15220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f277a978387143d89c960def5b77cf24. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.460003 15220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: LEARNER
W20260620 07:07:13.491614 15501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:13.491964 15501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:13.492030 15501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:13.496207 15501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:13.496260 15501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:13.496340 15501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:13.501425 15501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:36375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=36791
--tserver_master_addrs=127.8.24.62:42557,127.8.24.61:39679,127.8.24.60:41943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:13.502417 15501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:13.503363 15501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:13.512523 15506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.512555 15509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:13.512559 15507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:13.512594 15501 server_base.cc:1034] running on GCE node
I20260620 07:07:13.513200 15501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:13.513764 15501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:13.514905 15501 hybrid_clock.cc:648] HybridClock initialized: now 1781939233514885 us; error 41 us; skew 500 ppm
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:13.517366 15501 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260620 07:07:13.518206 15501 webserver.cc:460] Webserver started at http://127.8.24.3:36791/ using document root <none> and password file <none>
I20260620 07:07:13.518708 15501 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:13.518785 15501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:13.522382 15501 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:13.524233 15516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.525344 15501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:13.525440 15501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "ee8d07b71c6a4db69f08fd2eace1f92c"
format_stamp: "Formatted at 2026-06-20 07:07:12 on dist-test-slave-wjw3"
server_key: "035c812299dd7fafa9acd3d30bd024b5"
server_key_iv: "8f23b229cdf0f74301ddbc58fe391f5b"
server_key_version: "encryptionkey@0"
I20260620 07:07:13.525954 15501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:13.544113 15501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:13.546880 15501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:13.547036 15501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:13.547569 15501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:13.548347 15501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:13.548419 15501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.548488 15501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:13.548513 15501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:13.556407 15501 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:36375
I20260620 07:07:13.556455 15629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:36375 every 8 connection(s)
I20260620 07:07:13.557345 15501 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:13.560829 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 15501
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.60@KRBTEST.COM
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.61@KRBTEST.COM
Jun 20 07:07:13 dist-test-slave-wjw3 krb5kdc[14349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939233, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:13.574463 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.559426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:40437)
0620 07:07:13.560516 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.560529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.560860 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.561428 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.561436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.562056 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.562779 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.562795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563900 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.563903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.566498 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.566506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.567171 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.567175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.567265 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.567717 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.567730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.569753 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.572610 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.572613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.572615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.572839 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.573402 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.573405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.573407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.573494 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.573914 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.573917 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.574006 (+ 89us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.574116 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.574240 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":1681,"spinlock_wait_cycles":53504,"thread_start_us":158,"threads_started":1}
I20260620 07:07:13.574821 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.559427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.8.24.3:60455)
0620 07:07:13.559900 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.559949 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.560854 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.561603 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.561606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.561994 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.562777 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.562795 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563951 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.563954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.565137 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.565148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.565819 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.565823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.566030 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.566805 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.566821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.569075 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.571104 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.571110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.571129 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.571475 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.571783 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.571786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.571788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.571959 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.572372 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.572379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.572676 (+ 297us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.572850 (+ 174us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.573191 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":148,"spinlock_wait_cycles":11136,"thread_start_us":113,"threads_started":1}
I20260620 07:07:13.575423 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.558994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40437 (local address 127.8.24.62:42557)
0620 07:07:13.559134 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.559139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.560640 (+ 1501us) server_negotiation.cc:407] Connection header received
0620 07:07:13.561093 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.561097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.561159 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.561239 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.562939 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563763 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.566637 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.566957 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.567057 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.569893 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.569897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.569900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.569928 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.572503 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.573222 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.573225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.573227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.573304 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.573585 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.573589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.573591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.573789 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.573873 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.575149 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.575274 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 07:07:13.576749 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.560300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.8.24.3:36931)
0620 07:07:13.561214 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.561225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.561343 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.561689 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.561692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.562019 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:13.562774 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.562795 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563944 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.563947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.565427 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.565436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.566029 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.566032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.566134 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.566691 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:13.566711 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:13.569546 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:13.575253 (+ 5707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.575256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.575258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.575505 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.575783 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:13.575786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:13.575787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:13.575862 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:13.576286 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:13.576289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:13.576372 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.576458 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.576584 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":724,"spinlock_wait_cycles":2304,"thread_start_us":75,"threads_started":1}
I20260620 07:07:13.576800 15631 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42557
I20260620 07:07:13.577037 15631 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.577215 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.560438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:36931 (local address 127.8.24.60:41943)
0620 07:07:13.560547 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.560550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.561285 (+ 735us) server_negotiation.cc:407] Connection header received
0620 07:07:13.561437 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.561440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.561486 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.561535 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.563092 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563808 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.565542 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.565837 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.565915 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.570923 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.570927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.570929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.570952 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.574624 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.575620 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.575624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.575625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.575700 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.575955 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.575958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.575960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.576156 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.576245 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.576980 (+ 735us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.577079 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:13.577723 15630 heartbeater.cc:347] Connected to a master server at 127.8.24.60:41943
I20260620 07:07:13.577811 15630 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.577819 15631 heartbeater.cc:511] Master 127.8.24.62:42557 requested a full tablet report, sending...
I20260620 07:07:13.577998 15630 heartbeater.cc:511] Master 127.8.24.60:41943 requested a full tablet report, sending...
I20260620 07:07:13.578462 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.565889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.8.24.60:41943)
0620 07:07:13.569127 (+ 3238us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.569139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.569151 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:13.569186 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.569189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.569411 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.570840 (+ 1429us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.570928 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.571685 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.573088 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.573397 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.573793 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.574861 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.574865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.574867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.574890 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.576465 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.576938 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.576941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.576942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.577024 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.577309 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.577312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.577314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.577504 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.578147 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.578214 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.578317 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3162,"thread_start_us":79,"threads_started":1}
I20260620 07:07:13.578962 14394 ts_manager.cc:194] Re-registered known tserver with Master: ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:13.579037 15177 ts_manager.cc:194] Registered new tserver with Master: ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:13.579658 14394 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:40437
I20260620 07:07:13.579780 15643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.564591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.8.24.62:42557)
0620 07:07:13.565268 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.565272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.565286 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:07:13.565320 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.565323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.565366 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.565456 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.567485 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.568899 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.570280 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.570627 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.570729 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.572767 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.572771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.572773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.572798 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.576678 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.577622 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.577625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.577627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.577708 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.578145 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.578148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.578149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.578339 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.578460 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.579482 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.579602 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":75,"threads_started":1}
I20260620 07:07:13.579912 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.559029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60455 (local address 127.8.24.61:39679)
0620 07:07:13.559146 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.559150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.560054 (+ 904us) server_negotiation.cc:407] Connection header received
0620 07:07:13.561365 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.561368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.561418 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.561483 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.562938 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.563791 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.565263 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.565561 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.565661 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.569250 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.569254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.569256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.569286 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.570995 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.571596 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.571600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.571603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.571686 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.572064 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.572068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.572069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.572275 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.572359 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.572919 (+ 560us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.579763 (+ 6844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:07:13.580369 15632 heartbeater.cc:347] Connected to a master server at 127.8.24.61:39679
I20260620 07:07:13.580451 15632 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:13.580787 15632 heartbeater.cc:511] Master 127.8.24.61:39679 requested a full tablet report, sending...
I20260620 07:07:13.581413 14986 ts_manager.cc:194] Registered new tserver with Master: ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:13.582502 15641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.562913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.8.24.61:39679)
0620 07:07:13.563144 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.563148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.566250 (+ 3102us) server_negotiation.cc:407] Connection header received
0620 07:07:13.566292 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.566294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.566340 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.566403 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.567284 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.567969 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.570252 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.570521 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.570613 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.576482 (+ 5869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.576486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.576488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.576509 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.580334 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.580830 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.580833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.580835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.581183 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.581631 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.581634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.581636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.581905 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.582021 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.582297 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.582387 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
I20260620 07:07:13.582391 14394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:13.585449 14394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:13.587738 15144 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: NON_VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: true } } }
I20260620 07:07:13.587857 15144 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.590870 15648 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: attempting to promote NON_VOTER 263c09629cb347c389e73fea9441e609 to VOTER
I20260620 07:07:13.591264 15648 consensus_queue.cc:237] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:13.592253 14996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 07:07:13.592490 15188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 LEARNER]: Refusing update from remote peer f277a978387143d89c960def5b77cf24: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 07:07:13.592628 15649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 07:07:13.592845 15650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 07:07:13.595088 15648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 263c09629cb347c389e73fea9441e609 (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.595494 14996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 263c09629cb347c389e73fea9441e609 (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.595585 15188 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 263c09629cb347c389e73fea9441e609 (127.8.24.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.596330 15648 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.596431 15648 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.596592 15144 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.596695 15144 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.596652 15650 sys_catalog.cc:455] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.596740 15650 sys_catalog.cc:458] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:13.597484 15658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.597612 15658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:13.598214 15220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f277a978387143d89c960def5b77cf24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:13.598313 15220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:13.604840 15661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.599819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37015 (local address 127.0.0.1:34672)
0620 07:07:13.600284 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.600297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.600376 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.600954 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.600957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.600976 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.601086 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.601092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602466 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.602470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.603734 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.603742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604484 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.604488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.604568 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.604591 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.604643 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.604708 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":36,"thread_start_us":71,"threads_started":1}
I20260620 07:07:13.604894 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.600176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.8.24.1:37015)
0620 07:07:13.600479 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.600491 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.600512 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:07:13.600591 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.600597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.600812 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.600966 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.601198 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602346 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.603836 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604382 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.604475 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.604625 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.604685 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.604773 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":104,"threads_started":1}
I20260620 07:07:13.604990 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.599369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45503 (local address 127.0.0.1:46950)
0620 07:07:13.599685 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.599699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.599771 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.601339 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.601342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.601358 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.601432 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.601438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602332 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.602336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.603524 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.603535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604637 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.604641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.604741 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.604766 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.604810 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.604873 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":48}
I20260620 07:07:13.605378 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.600065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36375 (local address 127.0.0.1:44730)
0620 07:07:13.600525 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.600538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.600618 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.601203 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.601206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.601223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.601301 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.601307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602552 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.602555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.603760 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.603768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604976 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.605030 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.605111 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.605134 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.605201 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.605262 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":23,"thread_start_us":100,"threads_started":1}
I20260620 07:07:13.605489 15663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.600433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.8.24.3:36375)
0620 07:07:13.600702 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.600712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.600731 (+ 19us) server_negotiation.cc:407] Connection header received
0620 07:07:13.600791 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.600795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.600954 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.601082 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.601414 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602453 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.603874 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604714 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.604800 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.604922 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.605250 (+ 328us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.605345 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260620 07:07:13.605546 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.599669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.8.24.2:45503)
0620 07:07:13.600925 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.600936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.600955 (+ 19us) server_negotiation.cc:407] Connection header received
0620 07:07:13.601023 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.601028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.601229 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.601354 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.601514 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.602231 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.603653 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.604474 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.605161 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.605289 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.605336 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.605422 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":91,"threads_started":1}
I20260620 07:07:13.608144 15428 tablet_service.cc:1467] Processing CreateTablet for tablet 1aac60517aab438b9190f13e33e5ec19 (DEFAULT_TABLE table=test-table [id=f667bc059c1c4980996e88c1cf30574d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:13.608201 15564 tablet_service.cc:1467] Processing CreateTablet for tablet 1aac60517aab438b9190f13e33e5ec19 (DEFAULT_TABLE table=test-table [id=f667bc059c1c4980996e88c1cf30574d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:13.609015 15428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aac60517aab438b9190f13e33e5ec19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:13.609124 15564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aac60517aab438b9190f13e33e5ec19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:13.609100 15291 tablet_service.cc:1467] Processing CreateTablet for tablet 1aac60517aab438b9190f13e33e5ec19 (DEFAULT_TABLE table=test-table [id=f667bc059c1c4980996e88c1cf30574d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:13.609962 15291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aac60517aab438b9190f13e33e5ec19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:13.615262 15664 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Bootstrap starting.
I20260620 07:07:13.616294 15665 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Bootstrap starting.
I20260620 07:07:13.617143 15666 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Bootstrap starting.
I20260620 07:07:13.618263 15664 tablet_bootstrap.cc:654] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:13.618402 15665 tablet_bootstrap.cc:654] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:13.618887 15666 tablet_bootstrap.cc:654] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:13.619159 15665 log.cc:824] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:13.619159 15664 log.cc:824] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:13.619602 15666 log.cc:824] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:13.621131 15665 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: No bootstrap required, opened a new log
I20260620 07:07:13.621131 15664 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: No bootstrap required, opened a new log
I20260620 07:07:13.621313 15665 ts_tablet_manager.cc:1397] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:07:13.621313 15664 ts_tablet_manager.cc:1397] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 07:07:13.624900 15666 tablet_bootstrap.cc:492] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: No bootstrap required, opened a new log
I20260620 07:07:13.625079 15666 ts_tablet_manager.cc:1397] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260620 07:07:13.626094 15665 raft_consensus.cc:348] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.626365 15665 raft_consensus.cc:374] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:13.626427 15665 raft_consensus.cc:729] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8d07b71c6a4db69f08fd2eace1f92c, State: Initialized, Role: FOLLOWER
I20260620 07:07:13.626686 15664 raft_consensus.cc:348] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.626912 15664 raft_consensus.cc:374] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:13.626965 15664 raft_consensus.cc:729] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8c643e69c246ed8a84dd255c133453, State: Initialized, Role: FOLLOWER
I20260620 07:07:13.626900 15665 consensus_queue.cc:260] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.627368 15664 consensus_queue.cc:260] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.628065 15495 heartbeater.cc:503] Master 127.8.24.62:42557 was elected leader, sending a full tablet report...
I20260620 07:07:13.628624 15664 ts_tablet_manager.cc:1428] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260620 07:07:13.627912 15665 ts_tablet_manager.cc:1428] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260620 07:07:13.630046 15631 heartbeater.cc:503] Master 127.8.24.62:42557 was elected leader, sending a full tablet report...
I20260620 07:07:13.632591 15666 raft_consensus.cc:348] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.632836 15666 raft_consensus.cc:374] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:13.632903 15666 raft_consensus.cc:729] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c3a263a49b440990db588277f98150, State: Initialized, Role: FOLLOWER
I20260620 07:07:13.633292 15666 consensus_queue.cc:260] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.633764 15358 heartbeater.cc:503] Master 127.8.24.62:42557 was elected leader, sending a full tablet report...
I20260620 07:07:13.634056 15666 ts_tablet_manager.cc:1428] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Time spent starting tablet: real 0.009s user 0.000s sys 0.006s
W20260620 07:07:13.684419 15360 tablet.cc:2369] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:07:13.808594 15633 tablet.cc:2369] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:13.809178 15672 raft_consensus.cc:482] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:13.809370 15672 raft_consensus.cc:504] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.810492 15672 leader_election.cc:288] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503), ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:13.814316 15663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.811245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43445 (local address 127.8.24.3:36375)
0620 07:07:13.811357 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.811360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.811458 (+ 98us) server_negotiation.cc:407] Connection header received
0620 07:07:13.811606 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.811609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.811657 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.811725 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.811947 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.812442 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813404 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.813786 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813876 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.813932 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.814133 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.814201 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:07:13.814317 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.810874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55073 (local address 127.8.24.2:45503)
0620 07:07:13.811011 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.811015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.811108 (+ 93us) server_negotiation.cc:407] Connection header received
0620 07:07:13.811274 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.811278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.811332 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.811407 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:13.811718 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.812289 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813379 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.813776 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813860 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.813920 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.814151 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.814217 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:07:13.814332 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.811135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36375 (local address 127.8.24.1:43445)
0620 07:07:13.811377 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.811390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.811508 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.811741 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.811744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.811760 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.811853 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.811858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.812539 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.812541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.813286 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.813294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.813895 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.814014 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.814037 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.814098 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.814168 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":4,"spinlock_wait_cycles":4480,"thread_start_us":68,"threads_started":1}
I20260620 07:07:13.814342 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.810780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45503 (local address 127.8.24.1:55073)
0620 07:07:13.811021 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:13.811035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:13.811148 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:13.811475 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:13.811479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:13.811508 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:13.811612 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.811618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.812392 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.812396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.813260 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:13.813269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.813887 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.813891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:13.814024 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.814054 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 07:07:13.814122 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:07:13.814195 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":37,"spinlock_wait_cycles":768,"thread_start_us":91,"threads_started":1}
I20260620 07:07:13.814917 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aac60517aab438b9190f13e33e5ec19" candidate_uuid: "45c3a263a49b440990db588277f98150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" is_pre_election: true
I20260620 07:07:13.815068 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aac60517aab438b9190f13e33e5ec19" candidate_uuid: "45c3a263a49b440990db588277f98150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8c643e69c246ed8a84dd255c133453" is_pre_election: true
I20260620 07:07:13.815229 15584 raft_consensus.cc:2454] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c3a263a49b440990db588277f98150 in term 0.
I20260620 07:07:13.815290 15448 raft_consensus.cc:2454] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c3a263a49b440990db588277f98150 in term 0.
I20260620 07:07:13.815675 15245 leader_election.cc:302] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c3a263a49b440990db588277f98150, ee8d07b71c6a4db69f08fd2eace1f92c; no voters:
I20260620 07:07:13.815918 15672 raft_consensus.cc:2781] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:13.816008 15672 raft_consensus.cc:482] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:13.816040 15672 raft_consensus.cc:3037] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:13.816973 15672 raft_consensus.cc:504] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.817287 15672 leader_election.cc:288] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [CANDIDATE]: Term 1 election: Requested vote from peers 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503), ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375)
I20260620 07:07:13.817601 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aac60517aab438b9190f13e33e5ec19" candidate_uuid: "45c3a263a49b440990db588277f98150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8c643e69c246ed8a84dd255c133453"
I20260620 07:07:13.817644 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aac60517aab438b9190f13e33e5ec19" candidate_uuid: "45c3a263a49b440990db588277f98150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c"
I20260620 07:07:13.817740 15448 raft_consensus.cc:3037] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:13.817740 15584 raft_consensus.cc:3037] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:13.818962 15584 raft_consensus.cc:2454] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c3a263a49b440990db588277f98150 in term 1.
I20260620 07:07:13.818962 15448 raft_consensus.cc:2454] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c3a263a49b440990db588277f98150 in term 1.
I20260620 07:07:13.819223 15245 leader_election.cc:302] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c3a263a49b440990db588277f98150, ee8d07b71c6a4db69f08fd2eace1f92c; no voters:
I20260620 07:07:13.819378 15672 raft_consensus.cc:2781] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:13.819597 15672 raft_consensus.cc:686] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 1 LEADER]: Becoming Leader. State: Replica: 45c3a263a49b440990db588277f98150, State: Running, Role: LEADER
I20260620 07:07:13.819934 15672 consensus_queue.cc:237] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:13.823037 14393 catalog_manager.cc:5495] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c3a263a49b440990db588277f98150 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "45c3a263a49b440990db588277f98150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } health_report { overall_health: HEALTHY } } }
W20260620 07:07:13.869096 15497 tablet.cc:2369] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:13.889843 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.886879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.8.24.1:37015)
0620 07:07:13.886994 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.886998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.887011 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:13.887116 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.887119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.887174 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.887258 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:13.887464 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.887960 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.888697 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.888895 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.888959 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.889166 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:13.889556 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:13.889646 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.889671 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.889716 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:07:13.895224 15584 raft_consensus.cc:1261] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Refusing update from remote peer 45c3a263a49b440990db588277f98150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:13.895239 15448 raft_consensus.cc:1261] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Refusing update from remote peer 45c3a263a49b440990db588277f98150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:13.895874 15672 consensus_queue.cc:1035] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:13.896060 15675 consensus_queue.cc:1035] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:13.903870 15680 mvcc.cc:205] Tried to move back new op lower bound from 7298823102028410880 to 7298823101728870400. Current Snapshot: MvccSnapshot[applied={T|T < 7298823102028410880}]
I20260620 07:07:13.913934 14393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:13.914080 14393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:13.917822 14393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f277a978387143d89c960def5b77cf24: Sending DeleteTablet for 3 replicas of tablet 1aac60517aab438b9190f13e33e5ec19
I20260620 07:07:13.918612 15428 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aac60517aab438b9190f13e33e5ec19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:13 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46950
I20260620 07:07:13.918702 15564 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aac60517aab438b9190f13e33e5ec19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:13 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44730
I20260620 07:07:13.918992 15688 tablet_replica.cc:330] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: stopping tablet replica
I20260620 07:07:13.919113 15689 tablet_replica.cc:330] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: stopping tablet replica
I20260620 07:07:13.919296 15688 raft_consensus.cc:2229] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:13.919370 15689 raft_consensus.cc:2229] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:13.919601 15688 raft_consensus.cc:2258] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:13.919601 15689 raft_consensus.cc:2258] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:13.919690 15291 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aac60517aab438b9190f13e33e5ec19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:13 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:34672
I20260620 07:07:13.920975 15688 ts_tablet_manager.cc:1905] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:13.920974 15689 ts_tablet_manager.cc:1905] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:13.922899 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14362
I20260620 07:07:13.924190 15689 ts_tablet_manager.cc:1918] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:13.924238 15688 ts_tablet_manager.cc:1918] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:13.924281 15689 log.cc:1196] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/1aac60517aab438b9190f13e33e5ec19
I20260620 07:07:13.924302 15688 log.cc:1196] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/1aac60517aab438b9190f13e33e5ec19
I20260620 07:07:13.924589 15688 ts_tablet_manager.cc:1939] T 1aac60517aab438b9190f13e33e5ec19 P 5e8c643e69c246ed8a84dd255c133453: Deleting consensus metadata
I20260620 07:07:13.924589 15689 ts_tablet_manager.cc:1939] T 1aac60517aab438b9190f13e33e5ec19 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting consensus metadata
I20260620 07:07:13.927724 15690 tablet_replica.cc:330] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: stopping tablet replica
I20260620 07:07:13.927994 15690 raft_consensus.cc:2229] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:13.928299 15690 raft_consensus.cc:2258] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:07:13.929903 15519 connection.cc:535] server connection from 127.0.0.1:44730 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 07:07:13.930094 15384 connection.cc:535] server connection from 127.0.0.1:46950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 07:07:13.930228 15690 ts_tablet_manager.cc:1905] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:13.932943 15690 ts_tablet_manager.cc:1918] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:13.933043 15690 log.cc:1196] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/1aac60517aab438b9190f13e33e5ec19
I20260620 07:07:13.933396 15690 ts_tablet_manager.cc:1939] T 1aac60517aab438b9190f13e33e5ec19 P 45c3a263a49b440990db588277f98150: Deleting consensus metadata
W20260620 07:07:13.934419 15245 connection.cc:441] server connection from 127.0.0.1:34672 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34672 (request call id 1) could send its response
I20260620 07:07:13.939203 15641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.921427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.8.24.61:39679)
0620 07:07:13.921546 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.921550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.928708 (+ 7158us) server_negotiation.cc:407] Connection header received
0620 07:07:13.928898 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.928901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.928957 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.929028 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.931017 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.931802 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.933375 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.933705 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.933780 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.935687 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.935691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.935694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.935720 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.937433 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.937903 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.937906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.937908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.937983 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.938279 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.938282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.938283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.938496 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.938594 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.938919 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.939039 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":50}
I20260620 07:07:13.943423 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.934348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.8.24.60:41943)
0620 07:07:13.934501 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.934505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.934994 (+ 489us) server_negotiation.cc:407] Connection header received
0620 07:07:13.935122 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.935125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.935173 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.935239 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.935868 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.936651 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.937802 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.938100 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.938167 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.939949 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.939952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.939955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.939978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.941752 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.942177 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.942180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.942182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.942264 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.942520 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.942523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.942525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.942737 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.942826 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.943160 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.943268 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"spinlock_wait_cycles":1280}
I20260620 07:07:13.944660 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.933816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.8.24.61:39679)
0620 07:07:13.933914 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.933918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.935195 (+ 1277us) server_negotiation.cc:407] Connection header received
0620 07:07:13.935459 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.935462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.935501 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.935547 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.936428 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.937190 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.938367 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.938862 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.938942 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.941407 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.941411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.941413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.941437 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.942988 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.943426 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.943430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.943452 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.943530 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.943810 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.943814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.943816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.944014 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.944103 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.944411 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.944512 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":189,"server-negotiator.queue_time_us":36}
I20260620 07:07:13.947078 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:13.921826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.8.24.60:41943)
0620 07:07:13.921922 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:13.921926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:13.939544 (+ 17618us) server_negotiation.cc:407] Connection header received
0620 07:07:13.939712 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:13.939715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:13.939761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:13.939809 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:13.940516 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.941245 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.942275 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:13.942586 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:13.942659 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:13.944259 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:13.944262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:13.944264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:13.944286 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:13.945723 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.946070 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.946073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.946075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.946144 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:13.946366 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:13.946369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:13.946371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:13.946529 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:13.946632 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:13.946833 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 07:07:13.946931 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":36}
I20260620 07:07:14.907794 15699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:14.907324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:49363)
0620 07:07:14.907594 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:14.907688 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":67,"threads_started":1}
W20260620 07:07:14.908062 15631 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:42557 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:14.908489 15700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:14.907989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:47959)
0620 07:07:14.908314 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:14.908399 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":75,"threads_started":1}
W20260620 07:07:14.908995 15358 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:42557 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:14.909948 15701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:14.909632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:32923)
0620 07:07:14.909808 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:14.909870 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":52,"threads_started":1}
W20260620 07:07:14.910169 15495 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:42557 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:15.418134 15702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f277a978387143d89c960def5b77cf24)
I20260620 07:07:15.418330 15702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:15.418803 15703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f277a978387143d89c960def5b77cf24)
I20260620 07:07:15.418978 15703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:15.419337 15702 leader_election.cc:288] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943)
I20260620 07:07:15.419960 15703 leader_election.cc:288] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 995e53db55434dc9938a2720cc84e872 (127.8.24.61:39679)
I20260620 07:07:15.420040 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.419537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51958)
0620 07:07:15.419861 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.419940 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":58,"threads_started":1}
I20260620 07:07:15.420385 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.419996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51970)
0620 07:07:15.420227 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.420300 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":42,"threads_started":1}
W20260620 07:07:15.420560 14981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
W20260620 07:07:15.420567 15165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:15.421444 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.421216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51972)
0620 07:07:15.421338 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.421382 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260620 07:07:15.421505 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.421282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51980)
0620 07:07:15.421403 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.421444 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260620 07:07:15.421590 15165 leader_election.cc:334] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
W20260620 07:07:15.421645 14981 leader_election.cc:334] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:15.423712 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.420112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:39679 (local address 127.0.0.1:53454)
0620 07:07:15.420297 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.420309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:15.420400 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:15.420828 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:15.420831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:15.420849 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:15.420952 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:15.420958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.421962 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.421965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:15.422740 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:15.422748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.423338 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.423341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:15.423426 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:15.423468 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 07:07:15.423530 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:07:15.423597 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260620 07:07:15.423745 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.420356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.8.24.61:39679)
0620 07:07:15.420595 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:15.420600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:15.420612 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:15.420654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:15.420657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:15.420698 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:15.421050 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:15.421139 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.421866 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.422848 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.423244 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.423328 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:15.423462 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:15.423560 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 07:07:15.423637 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":171,"thread_start_us":62,"threads_started":1}
I20260620 07:07:15.423841 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.419801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.8.24.60:41943)
0620 07:07:15.420299 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:15.420304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:15.420320 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:07:15.420694 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:15.420697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:15.420742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:15.420824 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:15.421180 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.421897 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.422852 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.423251 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.423337 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:15.423392 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:15.423618 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 07:07:15.423683 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"spinlock_wait_cycles":1408,"thread_start_us":46,"threads_started":1}
I20260620 07:07:15.424034 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.419546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:41943 (local address 127.0.0.1:33766)
0620 07:07:15.419882 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.419897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:15.420564 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:15.420951 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:15.420954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:15.420969 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:15.421059 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:15.421066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.421995 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.421998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:15.422740 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:15.422748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:15.423349 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:15.423352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:15.423512 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:15.423529 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:07:15.423593 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:07:15.423667 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":57,"thread_start_us":58,"threads_started":1}
I20260620 07:07:15.424217 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "263c09629cb347c389e73fea9441e609" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "995e53db55434dc9938a2720cc84e872" is_pre_election: true
I20260620 07:07:15.424402 14996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 263c09629cb347c389e73fea9441e609 in term 1.
I20260620 07:07:15.424410 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995e53db55434dc9938a2720cc84e872" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "263c09629cb347c389e73fea9441e609" is_pre_election: true
I20260620 07:07:15.424567 15188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 995e53db55434dc9938a2720cc84e872 in term 1.
I20260620 07:07:15.424737 15164 leader_election.cc:302] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 263c09629cb347c389e73fea9441e609, 995e53db55434dc9938a2720cc84e872; no voters: f277a978387143d89c960def5b77cf24
I20260620 07:07:15.424824 14980 leader_election.cc:302] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 263c09629cb347c389e73fea9441e609, 995e53db55434dc9938a2720cc84e872; no voters: f277a978387143d89c960def5b77cf24
I20260620 07:07:15.424932 15703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 07:07:15.424983 15703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f277a978387143d89c960def5b77cf24)
I20260620 07:07:15.424988 15702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 07:07:15.425020 15703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:07:15.425030 15702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f277a978387143d89c960def5b77cf24)
I20260620 07:07:15.425067 15702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:07:15.426170 15703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:15.426170 15702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:15.426506 15703 leader_election.cc:288] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 election: Requested vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 995e53db55434dc9938a2720cc84e872 (127.8.24.61:39679)
I20260620 07:07:15.426505 15702 leader_election.cc:288] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 election: Requested vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943)
I20260620 07:07:15.426761 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.426592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51988)
0620 07:07:15.426669 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.426712 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 07:07:15.426949 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "263c09629cb347c389e73fea9441e609" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "995e53db55434dc9938a2720cc84e872"
I20260620 07:07:15.427018 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995e53db55434dc9938a2720cc84e872" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "263c09629cb347c389e73fea9441e609"
I20260620 07:07:15.427099 14996 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 263c09629cb347c389e73fea9441e609 in current term 2: Already voted for candidate 995e53db55434dc9938a2720cc84e872 in this term.
I20260620 07:07:15.427111 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.426809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:51998)
0620 07:07:15.427012 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.427061 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260620 07:07:15.427163 15188 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 995e53db55434dc9938a2720cc84e872 in current term 2: Already voted for candidate 263c09629cb347c389e73fea9441e609 in this term.
I20260620 07:07:15.427358 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.427065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:52002)
0620 07:07:15.427181 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.427306 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260620 07:07:15.427479 14981 leader_election.cc:334] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:15.427541 14981 leader_election.cc:302] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 995e53db55434dc9938a2720cc84e872; no voters: 263c09629cb347c389e73fea9441e609, f277a978387143d89c960def5b77cf24
I20260620 07:07:15.427724 15702 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 07:07:15.427817 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.427605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:52006)
0620 07:07:15.427724 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.427768 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260620 07:07:15.427899 15165 leader_election.cc:334] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:15.427963 15165 leader_election.cc:302] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 263c09629cb347c389e73fea9441e609; no voters: 995e53db55434dc9938a2720cc84e872, f277a978387143d89c960def5b77cf24
I20260620 07:07:15.428098 15703 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 07:07:15.908999 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.908576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:43195)
0620 07:07:15.908844 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.908911 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
I20260620 07:07:15.909971 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.909570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:55713)
0620 07:07:15.909836 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.909893 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
I20260620 07:07:15.910929 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:15.910604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:44343)
0620 07:07:15.910795 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:15.910848 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260620 07:07:16.910099 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:16.909671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:42699)
0620 07:07:16.909919 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:16.909993 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260620 07:07:16.911057 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:16.910700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:49603)
0620 07:07:16.910912 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:16.910972 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260620 07:07:16.911783 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:16.911475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:47079)
0620 07:07:16.911669 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:16.911716 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":54,"threads_started":1}
I20260620 07:07:17.443334 15718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:17.443488 15718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:17.443890 15718 leader_election.cc:288] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943)
I20260620 07:07:17.444344 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995e53db55434dc9938a2720cc84e872" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "263c09629cb347c389e73fea9441e609" is_pre_election: true
I20260620 07:07:17.444478 15188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 995e53db55434dc9938a2720cc84e872 in term 2.
I20260620 07:07:17.444512 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.444065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55742)
0620 07:07:17.444355 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.444408 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":79,"threads_started":1}
I20260620 07:07:17.444783 14980 leader_election.cc:302] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263c09629cb347c389e73fea9441e609, 995e53db55434dc9938a2720cc84e872; no voters:
I20260620 07:07:17.444931 15718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260620 07:07:17.444998 15718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:17.445070 15718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 2 FOLLOWER]: Advancing to term 3
I20260620 07:07:17.445237 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.445009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55758)
0620 07:07:17.445118 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.445176 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 07:07:17.445333 14981 leader_election.cc:334] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:17.446048 15718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:17.446336 15718 leader_election.cc:288] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 election: Requested vote from peers f277a978387143d89c960def5b77cf24 (127.8.24.62:42557), 263c09629cb347c389e73fea9441e609 (127.8.24.60:41943)
I20260620 07:07:17.446632 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.446435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55774)
0620 07:07:17.446522 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.446580 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 07:07:17.446645 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "995e53db55434dc9938a2720cc84e872" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "263c09629cb347c389e73fea9441e609"
I20260620 07:07:17.446767 15188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 2 FOLLOWER]: Advancing to term 3
I20260620 07:07:17.447209 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.447033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55778)
0620 07:07:17.447124 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.447165 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 07:07:17.447292 14981 leader_election.cc:334] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
I20260620 07:07:17.447788 15188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 995e53db55434dc9938a2720cc84e872 in term 3.
I20260620 07:07:17.448035 14980 leader_election.cc:302] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 263c09629cb347c389e73fea9441e609, 995e53db55434dc9938a2720cc84e872; no voters: f277a978387143d89c960def5b77cf24
I20260620 07:07:17.448175 15718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 07:07:17.448328 15718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [term 3 LEADER]: Becoming Leader. State: Replica: 995e53db55434dc9938a2720cc84e872, State: Running, Role: LEADER
I20260620 07:07:17.448552 15718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } }
I20260620 07:07:17.449496 15721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995e53db55434dc9938a2720cc84e872. Latest consensus state: current_term: 3 leader_uuid: "995e53db55434dc9938a2720cc84e872" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:17.449611 15721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:17.449896 15723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:17.452026 15723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:17.452382 15723 catalog_manager.cc:1259] Loaded cluster ID: 45ad0780c6dc4b90b9329ca783f0f075
I20260620 07:07:17.452422 15723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:17.453025 15723 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:17.453375 15723 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Loaded TSK: 0
I20260620 07:07:17.454010 15723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:17.463968 14986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:17.466765 14986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:17.469380 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.469153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55808)
0620 07:07:17.469257 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.469310 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 07:07:17.469439 15188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [term 3 FOLLOWER]: Refusing update from remote peer 995e53db55434dc9938a2720cc84e872: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260620 07:07:17.469853 15718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 07:07:17.469976 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.469788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.0.0.1:55824)
0620 07:07:17.469883 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.469925 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 07:07:17.470067 14981 consensus_peers.cc:487] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 -> Peer f277a978387143d89c960def5b77cf24 (127.8.24.62:42557): Couldn't send request to peer f277a978387143d89c960def5b77cf24. Status: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 07:07:17.471128 15725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 995e53db55434dc9938a2720cc84e872. Latest consensus state: current_term: 3 leader_uuid: "995e53db55434dc9938a2720cc84e872" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:17.471244 15725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:17.472221 15721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "995e53db55434dc9938a2720cc84e872" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:17.472329 15721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:17.472726 15725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "995e53db55434dc9938a2720cc84e872" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f277a978387143d89c960def5b77cf24" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42557 } } peers { permanent_uuid: "995e53db55434dc9938a2720cc84e872" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 39679 } attrs { promote: false } } peers { permanent_uuid: "263c09629cb347c389e73fea9441e609" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 41943 } attrs { promote: false } } }
I20260620 07:07:17.472872 15725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c09629cb347c389e73fea9441e609 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:07:17.481599 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.477338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45503 (local address 127.0.0.1:58732)
0620 07:07:17.477438 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.477452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:17.477533 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:17.478564 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:17.478567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:17.478582 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:17.478644 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.478648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479501 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.479504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.480414 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.480423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.481236 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.481321 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.481342 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:07:17.481405 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 07:07:17.481472 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":2}
I20260620 07:07:17.481611 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.477465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.8.24.2:45503)
0620 07:07:17.478365 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.478369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.478381 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:17.478426 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.478429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.478465 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.478527 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:17.478747 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479358 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.480550 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.481137 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481229 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.481288 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.481428 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.481501 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":90,"threads_started":1}
I20260620 07:07:17.481876 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.477527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:36375 (local address 127.0.0.1:45430)
0620 07:07:17.477893 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.477906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:17.477984 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:17.478371 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:17.478374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:17.478390 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:17.478480 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.478484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479430 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.479462 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.480554 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.480564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.481486 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.481588 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.481609 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:07:17.481671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:07:17.481739 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":29,"thread_start_us":79,"threads_started":1}
I20260620 07:07:17.482043 15428 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc08939db4e9398207ca6c6f846f5 (DEFAULT_TABLE table=test-table [id=ab6af507eba44886888d787900cc6c34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:17.482224 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.477827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.8.24.3:36375)
0620 07:07:17.478066 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.478071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.478084 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:17.478130 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.478133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.478179 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.478258 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:17.478598 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479327 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.480691 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.481285 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481377 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.481436 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.482018 (+ 582us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.482099 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260620 07:07:17.482327 15428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc08939db4e9398207ca6c6f846f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:17.482579 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.477789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37015 (local address 127.0.0.1:48060)
0620 07:07:17.477991 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.478002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:17.478065 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:17.478929 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:17.478931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:17.478948 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:17.479023 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.479028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479931 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.479934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.480990 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.480997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.482063 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.482067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.482145 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.482167 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:07:17.482221 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 07:07:17.482283 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260620 07:07:17.482669 15564 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc08939db4e9398207ca6c6f846f5 (DEFAULT_TABLE table=test-table [id=ab6af507eba44886888d787900cc6c34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:17.482909 15564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc08939db4e9398207ca6c6f846f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:17.482913 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.478192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.8.24.1:37015)
0620 07:07:17.478726 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.478730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.478741 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:17.478785 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.478788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.478830 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.478908 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:17.479120 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.479823 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481124 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.481786 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.481867 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.481926 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.482722 (+ 796us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.482797 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":64,"threads_started":1}
I20260620 07:07:17.483703 15291 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc08939db4e9398207ca6c6f846f5 (DEFAULT_TABLE table=test-table [id=ab6af507eba44886888d787900cc6c34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:17.483942 15291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc08939db4e9398207ca6c6f846f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:17.485085 15737 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Bootstrap starting.
I20260620 07:07:17.485991 15737 tablet_bootstrap.cc:654] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:17.486240 15738 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150: Bootstrap starting.
I20260620 07:07:17.486727 15736 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Bootstrap starting.
I20260620 07:07:17.487030 15738 tablet_bootstrap.cc:654] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:17.487489 15736 tablet_bootstrap.cc:654] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:17.488006 15738 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150: No bootstrap required, opened a new log
I20260620 07:07:17.488090 15738 ts_tablet_manager.cc:1397] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:17.488101 15737 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: No bootstrap required, opened a new log
I20260620 07:07:17.488169 15737 ts_tablet_manager.cc:1397] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 07:07:17.488451 15738 raft_consensus.cc:348] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.488550 15738 raft_consensus.cc:374] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:17.488581 15738 raft_consensus.cc:729] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c3a263a49b440990db588277f98150, State: Initialized, Role: FOLLOWER
I20260620 07:07:17.488618 15736 tablet_bootstrap.cc:492] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: No bootstrap required, opened a new log
I20260620 07:07:17.488569 15737 raft_consensus.cc:348] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.488665 15737 raft_consensus.cc:374] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:17.488701 15737 raft_consensus.cc:729] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8d07b71c6a4db69f08fd2eace1f92c, State: Initialized, Role: FOLLOWER
I20260620 07:07:17.488737 15736 ts_tablet_manager.cc:1397] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:17.488725 15738 consensus_queue.cc:260] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.488808 15737 consensus_queue.cc:260] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.489013 15738 ts_tablet_manager.cc:1428] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:17.489116 15737 ts_tablet_manager.cc:1428] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:17.489120 15736 raft_consensus.cc:348] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.489652 15736 raft_consensus.cc:374] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:17.489703 15736 raft_consensus.cc:729] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8c643e69c246ed8a84dd255c133453, State: Initialized, Role: FOLLOWER
I20260620 07:07:17.491272 15736 consensus_queue.cc:260] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.491619 15359 heartbeater.cc:503] Master 127.8.24.61:39679 was elected leader, sending a full tablet report...
I20260620 07:07:17.491783 15736 ts_tablet_manager.cc:1428] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260620 07:07:17.492964 15632 heartbeater.cc:503] Master 127.8.24.61:39679 was elected leader, sending a full tablet report...
I20260620 07:07:17.493299 15496 heartbeater.cc:503] Master 127.8.24.61:39679 was elected leader, sending a full tablet report...
I20260620 07:07:17.493851 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.489480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:35869)
0620 07:07:17.493705 (+ 4225us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.493762 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4162,"thread_start_us":94,"threads_started":1}
I20260620 07:07:17.494333 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.489836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:55395)
0620 07:07:17.494191 (+ 4355us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.494242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4292,"thread_start_us":80,"threads_started":1}
I20260620 07:07:17.494685 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.494254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:58739)
0620 07:07:17.494563 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.494619 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":84,"threads_started":1}
I20260620 07:07:17.495221 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.492003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:54047)
0620 07:07:17.495090 (+ 3087us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.495135 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3027,"thread_start_us":95,"threads_started":1}
I20260620 07:07:17.495785 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.495611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:45779)
0620 07:07:17.495699 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.495739 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 07:07:17.647503 15743 raft_consensus.cc:482] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:17.647658 15743 raft_consensus.cc:504] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.648725 15743 leader_election.cc:288] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503), 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
I20260620 07:07:17.652362 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.648980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37015 (local address 127.8.24.3:43133)
0620 07:07:17.649190 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.649203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:17.649311 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:17.649545 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:17.649548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:17.649571 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:17.649719 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.649724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.650508 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.650511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.651305 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.651313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.651977 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.652084 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.652101 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:07:17.652169 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:07:17.652230 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":24,"spinlock_wait_cycles":11392}
I20260620 07:07:17.652354 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.649030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47747 (local address 127.8.24.2:45503)
0620 07:07:17.649158 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.649162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.649245 (+ 83us) server_negotiation.cc:407] Connection header received
0620 07:07:17.649410 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.649412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.649452 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.649509 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:17.649829 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.650325 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651415 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.651842 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651929 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.651987 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.652167 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.652231 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 07:07:17.652366 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.649085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:43133 (local address 127.8.24.1:37015)
0620 07:07:17.649196 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.649201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.649275 (+ 74us) server_negotiation.cc:407] Connection header received
0620 07:07:17.649412 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.649415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.649457 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.649521 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:17.649826 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.650406 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651426 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.651849 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651943 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.652001 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.652199 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.652267 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:17.652383 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.648953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:45503 (local address 127.8.24.3:47747)
0620 07:07:17.649151 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.649164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:17.649312 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:17.649530 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:17.649533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:17.649569 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:17.649727 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.649732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.650416 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.650419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.651294 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:17.651303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.651963 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.651967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:17.652055 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.652077 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:07:17.652135 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 07:07:17.652198 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":67,"spinlock_wait_cycles":18560}
I20260620 07:07:17.652848 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc08939db4e9398207ca6c6f846f5" candidate_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8c643e69c246ed8a84dd255c133453" is_pre_election: true
I20260620 07:07:17.652920 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc08939db4e9398207ca6c6f846f5" candidate_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c3a263a49b440990db588277f98150" is_pre_election: true
I20260620 07:07:17.652989 15448 raft_consensus.cc:2454] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8d07b71c6a4db69f08fd2eace1f92c in term 0.
I20260620 07:07:17.653142 15311 raft_consensus.cc:2454] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8d07b71c6a4db69f08fd2eace1f92c in term 0.
I20260620 07:07:17.653359 15518 leader_election.cc:302] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e8c643e69c246ed8a84dd255c133453, ee8d07b71c6a4db69f08fd2eace1f92c; no voters:
I20260620 07:07:17.653611 15743 raft_consensus.cc:2781] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:17.653671 15743 raft_consensus.cc:482] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:17.653717 15743 raft_consensus.cc:3037] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:17.654677 15743 raft_consensus.cc:504] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.654968 15743 leader_election.cc:288] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [CANDIDATE]: Term 1 election: Requested vote from peers 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503), 45c3a263a49b440990db588277f98150 (127.8.24.1:37015)
I20260620 07:07:17.655258 15448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc08939db4e9398207ca6c6f846f5" candidate_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e8c643e69c246ed8a84dd255c133453"
I20260620 07:07:17.655282 15311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc08939db4e9398207ca6c6f846f5" candidate_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c3a263a49b440990db588277f98150"
I20260620 07:07:17.655357 15448 raft_consensus.cc:3037] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:17.655386 15311 raft_consensus.cc:3037] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:17.656214 15448 raft_consensus.cc:2454] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8d07b71c6a4db69f08fd2eace1f92c in term 1.
I20260620 07:07:17.656286 15311 raft_consensus.cc:2454] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8d07b71c6a4db69f08fd2eace1f92c in term 1.
I20260620 07:07:17.656445 15518 leader_election.cc:302] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e8c643e69c246ed8a84dd255c133453, ee8d07b71c6a4db69f08fd2eace1f92c; no voters:
I20260620 07:07:17.656608 15743 raft_consensus.cc:2781] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:17.656821 15743 raft_consensus.cc:686] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 LEADER]: Becoming Leader. State: Replica: ee8d07b71c6a4db69f08fd2eace1f92c, State: Running, Role: LEADER
I20260620 07:07:17.657056 15743 consensus_queue.cc:237] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } }
I20260620 07:07:17.657388 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.657204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:59585)
0620 07:07:17.657284 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.657332 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 07:07:17.659286 14994 catalog_manager.cc:5495] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c reported cstate change: term changed from 0 to 1, leader changed from <none> to ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3). New cstate: current_term: 1 leader_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee8d07b71c6a4db69f08fd2eace1f92c" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:17.669808 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.666972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.8.24.3:36375)
0620 07:07:17.667067 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:17.667072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:17.667084 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:17.667188 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:17.667196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:17.667240 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:17.667318 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:17.667498 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.667986 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.668724 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:17.668930 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:17.669004 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:17.669168 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:17.669574 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:17.669649 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:17.669679 (+ 30us) server_negotiation.cc:299] Negotiation successful
0620 07:07:17.669720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 07:07:17.672070 15448 raft_consensus.cc:1261] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Refusing update from remote peer ee8d07b71c6a4db69f08fd2eace1f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:17.672312 15311 raft_consensus.cc:1261] T 12abc08939db4e9398207ca6c6f846f5 P 45c3a263a49b440990db588277f98150 [term 1 FOLLOWER]: Refusing update from remote peer ee8d07b71c6a4db69f08fd2eace1f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:17.672515 15749 consensus_queue.cc:1035] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e8c643e69c246ed8a84dd255c133453" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:17.672778 15743 consensus_queue.cc:1035] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c3a263a49b440990db588277f98150" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:17.674355 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.673885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:43003)
0620 07:07:17.674230 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.674285 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
I20260620 07:07:17.675323 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.675080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:57391)
0620 07:07:17.675212 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.675260 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260620 07:07:17.677938 15755 mvcc.cc:205] Tried to move back new op lower bound from 7298823117500588032 to 7298823117445840896. Current Snapshot: MvccSnapshot[applied={T|T < 7298823117500588032}]
I20260620 07:07:17.678051 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.677833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.3:34803)
0620 07:07:17.677938 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.677989 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 07:07:17.678313 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.677833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.1:57239)
0620 07:07:17.678206 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.678253 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
I20260620 07:07:17.678704 15753 mvcc.cc:205] Tried to move back new op lower bound from 7298823117500588032 to 7298823117445840896. Current Snapshot: MvccSnapshot[applied={T|T < 7298823117500588032}]
I20260620 07:07:17.679855 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.679614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42557 (local address 127.8.24.2:34693)
0620 07:07:17.679749 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.679797 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260620 07:07:17.685823 14994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:17.685950 14994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:17.689306 14994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 995e53db55434dc9938a2720cc84e872: Sending DeleteTablet for 3 replicas of tablet 12abc08939db4e9398207ca6c6f846f5
I20260620 07:07:17.689970 15564 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc08939db4e9398207ca6c6f846f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:17 UTC) from {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:45430
I20260620 07:07:17.689975 15428 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc08939db4e9398207ca6c6f846f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:17 UTC) from {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:58732
I20260620 07:07:17.690189 15291 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc08939db4e9398207ca6c6f846f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:17 UTC) from {username='oryx', principal='oryx/127.8.24.61@KRBTEST.COM'} at 127.0.0.1:48060
I20260620 07:07:17.690411 15763 tablet_replica.cc:330] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: stopping tablet replica
I20260620 07:07:17.690517 15763 raft_consensus.cc:2229] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:17.690639 15763 raft_consensus.cc:2258] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:17.690724 15762 tablet_replica.cc:330] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: stopping tablet replica
I20260620 07:07:17.690815 15762 raft_consensus.cc:2229] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:17.690989 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 15138
I20260620 07:07:17.691223 15762 raft_consensus.cc:2258] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:17.691955 15762 ts_tablet_manager.cc:1905] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:17.694118 15762 ts_tablet_manager.cc:1918] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:17.694186 15762 log.cc:1196] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/12abc08939db4e9398207ca6c6f846f5
I20260620 07:07:17.694463 15762 ts_tablet_manager.cc:1939] T 12abc08939db4e9398207ca6c6f846f5 P ee8d07b71c6a4db69f08fd2eace1f92c: Deleting consensus metadata
I20260620 07:07:17.695056 14979 catalog_manager.cc:4841] TS ee8d07b71c6a4db69f08fd2eace1f92c (127.8.24.3:36375): tablet 12abc08939db4e9398207ca6c6f846f5 (table test-table [id=ab6af507eba44886888d787900cc6c34]) successfully deleted
I20260620 07:07:17.695266 15763 ts_tablet_manager.cc:1905] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:17.697052 15763 ts_tablet_manager.cc:1918] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:17.697119 15763 log.cc:1196] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/12abc08939db4e9398207ca6c6f846f5
I20260620 07:07:17.697340 15763 ts_tablet_manager.cc:1939] T 12abc08939db4e9398207ca6c6f846f5 P 5e8c643e69c246ed8a84dd255c133453: Deleting consensus metadata
I20260620 07:07:17.697889 14979 catalog_manager.cc:4841] TS 5e8c643e69c246ed8a84dd255c133453 (127.8.24.2:45503): tablet 12abc08939db4e9398207ca6c6f846f5 (table test-table [id=ab6af507eba44886888d787900cc6c34]) successfully deleted
I20260620 07:07:17.701709 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 15365
I20260620 07:07:17.701860 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:17.701622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37015 (local address 127.0.0.1:48072)
0620 07:07:17.701735 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:17.701790 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 07:07:17.702672 14981 catalog_manager.cc:4584] TS 45c3a263a49b440990db588277f98150 (127.8.24.1:37015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12abc08939db4e9398207ca6c6f846f5: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37015: connect: Connection refused (error 111)
I20260620 07:07:17.706707 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 15501
I20260620 07:07:17.711377 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14858
I20260620 07:07:17.715771 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 14966
2026-06-20T07:07:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7202 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:17 dist-test-slave-wjw3 krb5kdc[15767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:17 dist-test-slave-wjw3 krb5kdc[15767](info): set up 2 sockets
Jun 20 07:07:17 dist-test-slave-wjw3 krb5kdc[15767](info): commencing operation
krb5kdc: starting...
W20260620 07:07:18.659698 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:18 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
Entry for principal oryx/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab.
I20260620 07:07:18.765851 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43017
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:18.873870 15780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:18.874114 15780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:18.874151 15780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:18.878065 15780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:18.878119 15780 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 07:07:18.878135 15780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:18.878150 15780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:18.878168 15780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:18.878185 15780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:18.883239 15780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43017
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.62.keytab
--principal=oryx/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:18.884243 15780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:18.885179 15780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:18.893105 15786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:18.893110 15788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:18.893082 15785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:18.893224 15780 server_base.cc:1034] running on GCE node
I20260620 07:07:18.893878 15780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:18.894958 15780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:18.896113 15780 hybrid_clock.cc:648] HybridClock initialized: now 1781939238896086 us; error 38 us; skew 500 ppm
Jun 20 07:07:18 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939238, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:18.898613 15780 init.cc:377] Logged in from keytab as oryx/127.8.24.62@KRBTEST.COM (short username oryx)
I20260620 07:07:18.899457 15780 webserver.cc:460] Webserver started at http://127.8.24.62:33699/ using document root <none> and password file <none>
I20260620 07:07:18.899962 15780 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:18.900027 15780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:18.900205 15780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:18.902158 15780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "271425da39a043d0bd969564f273bd3b"
format_stamp: "Formatted at 2026-06-20 07:07:18 on dist-test-slave-wjw3"
server_key: "b6b840f37c11d833709706d3f0144087"
server_key_iv: "797752fb591aa959b7b7c0877cb6ee45"
server_key_version: "encryptionkey@0"
I20260620 07:07:18.902689 15780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "271425da39a043d0bd969564f273bd3b"
format_stamp: "Formatted at 2026-06-20 07:07:18 on dist-test-slave-wjw3"
server_key: "b6b840f37c11d833709706d3f0144087"
server_key_iv: "797752fb591aa959b7b7c0877cb6ee45"
server_key_version: "encryptionkey@0"
I20260620 07:07:18.906172 15780 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:07:18.908401 15795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:18.909539 15780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:18.909643 15780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "271425da39a043d0bd969564f273bd3b"
format_stamp: "Formatted at 2026-06-20 07:07:18 on dist-test-slave-wjw3"
server_key: "b6b840f37c11d833709706d3f0144087"
server_key_iv: "797752fb591aa959b7b7c0877cb6ee45"
server_key_version: "encryptionkey@0"
I20260620 07:07:18.909730 15780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:18.927037 15780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:18.929665 15780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:18.929833 15780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:18.937704 15780 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43017
I20260620 07:07:18.937712 15857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43017 every 8 connection(s)
I20260620 07:07:18.938637 15780 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:18.940907 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 15780
I20260620 07:07:18.941035 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:18.941339 8288 external_mini_cluster.cc:1428] Setting key 9c926ad9563bf2195abd2cf9da3e6aad
I20260620 07:07:18.941654 15858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:18.947824 15858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Bootstrap starting.
Jun 20 07:07:18 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:18.950104 15858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:18.950933 15858 log.cc:824] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:18.952919 15858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: No bootstrap required, opened a new log
I20260620 07:07:18.955610 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:18.942687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.8.24.62:43017)
0620 07:07:18.943100 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:18.943120 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:18.943312 (+ 192us) server_negotiation.cc:407] Connection header received
0620 07:07:18.944156 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:18.944174 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:18.944524 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:18.944833 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:18.945630 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:18.946631 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:18.947476 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:18.947978 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:18.948122 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:18.950552 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:18.950557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:18.950571 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:18.950599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:18.952648 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:18.953114 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:18.953119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:18.953124 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:18.953238 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:18.953567 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:18.953570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:18.953571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:18.954011 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:18.954164 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:18.954495 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 07:07:18.954783 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260620 07:07:18.956306 15858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } }
I20260620 07:07:18.956540 15858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:18.956650 15858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271425da39a043d0bd969564f273bd3b, State: Initialized, Role: FOLLOWER
I20260620 07:07:18.957136 15858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } }
I20260620 07:07:18.957293 15858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:18.957352 15858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:18.957448 15858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:18.958403 15858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } }
I20260620 07:07:18.958707 15858 leader_election.cc:302] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271425da39a043d0bd969564f273bd3b; no voters:
I20260620 07:07:18.958972 15858 leader_election.cc:288] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:18.959081 15863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:18.959287 15863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [term 1 LEADER]: Becoming Leader. State: Replica: 271425da39a043d0bd969564f273bd3b, State: Running, Role: LEADER
I20260620 07:07:18.959697 15863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } }
I20260620 07:07:18.960184 15858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:18.961410 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271425da39a043d0bd969564f273bd3b. Latest consensus state: current_term: 1 leader_uuid: "271425da39a043d0bd969564f273bd3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } } }
I20260620 07:07:18.961431 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271425da39a043d0bd969564f273bd3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271425da39a043d0bd969564f273bd3b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43017 } } }
I20260620 07:07:18.961541 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:18.961541 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:18.961941 15873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:18.964794 15873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:18.970491 15873 catalog_manager.cc:1347] Generated new cluster ID: 7d6cd38fb28e42d9a33a327008fd1894
I20260620 07:07:18.970558 15873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:18.983489 15873 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:18.984513 15873 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:18.989554 15873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Generated new TSK 0
I20260620 07:07:18.990305 15873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
I20260620 07:07:19.015882 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43017
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:19.129194 15884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:19.129477 15884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:19.129541 15884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:19.133481 15884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:19.133533 15884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:19.133589 15884 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:07:19.133641 15884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:19.138762 15884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:19.139792 15884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:19.140789 15884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:19.149741 15889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:19.149751 15890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.149801 15884 server_base.cc:1034] running on GCE node
W20260620 07:07:19.149803 15892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.150302 15884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:19.150853 15884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:19.151995 15884 hybrid_clock.cc:648] HybridClock initialized: now 1781939239151974 us; error 32 us; skew 500 ppm
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:19.154413 15884 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260620 07:07:19.155223 15884 webserver.cc:460] Webserver started at http://127.8.24.1:36581/ using document root <none> and password file <none>
I20260620 07:07:19.155764 15884 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:19.155826 15884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:19.156020 15884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:19.157920 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "32623289074e4faeb18eef1829883885"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "bed13f191f5c28ec72da0cc2b3ec99cc"
server_key_iv: "253d2c2fece06b79a0c92610dd0fbc35"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.158430 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "32623289074e4faeb18eef1829883885"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "bed13f191f5c28ec72da0cc2b3ec99cc"
server_key_iv: "253d2c2fece06b79a0c92610dd0fbc35"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.162079 15884 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:07:19.164408 15899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.165563 15884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:19.165661 15884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "32623289074e4faeb18eef1829883885"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "bed13f191f5c28ec72da0cc2b3ec99cc"
server_key_iv: "253d2c2fece06b79a0c92610dd0fbc35"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.165751 15884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:19.175751 15884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:19.178159 15884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:19.178313 15884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:19.179636 15884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:19.179690 15884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.179742 15884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:19.179769 15884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:19.189728 15884 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44527
I20260620 07:07:19.190038 16014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44527 every 8 connection(s)
I20260620 07:07:19.190660 15884 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:19.191910 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 15884
I20260620 07:07:19.192029 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:19.192358 8288 external_mini_cluster.cc:1428] Setting key 94fb1533357602c658f026e899c6b3e6
I20260620 07:07:19.192667 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.180269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:54221 (local address 127.8.24.62:43017)
0620 07:07:19.180417 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.180422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.181253 (+ 831us) server_negotiation.cc:407] Connection header received
0620 07:07:19.182430 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.182434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.182497 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.182556 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.184322 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.184998 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.186081 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.186319 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.186405 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.188884 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.188888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.188890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.188915 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.190440 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.190996 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.191000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.191002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.191109 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.191496 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.191500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.191501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.191738 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.191821 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.192389 (+ 568us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.192543 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 07:07:19.193598 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.180624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.1:54221)
0620 07:07:19.181088 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.181142 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.182175 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.182996 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.183006 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.183477 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.184148 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.184168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.185114 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.185118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.185971 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.185982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.186467 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.186472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.186626 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.187157 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.187172 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.188724 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.190550 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.190555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.190568 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.190892 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.191214 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.191217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.191219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.191375 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.191834 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.191840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.192140 (+ 300us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.192337 (+ 197us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.192664 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
I20260620 07:07:19.194766 16015 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43017
I20260620 07:07:19.194913 16015 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:19.195494 16015 heartbeater.cc:511] Master 127.8.24.62:43017 requested a full tablet report, sending...
I20260620 07:07:19.196847 15811 ts_manager.cc:194] Registered new tserver with Master: 32623289074e4faeb18eef1829883885 (127.8.24.1:44527)
I20260620 07:07:19.197943 15811 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:54221
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:19.206497 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.199817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.1:40051)
0620 07:07:19.200126 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.200138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.200215 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.200429 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.200430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.200582 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.200675 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.200680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.201668 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.201672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.202331 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.202341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.202758 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.202761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.202862 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.203315 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.203326 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.203686 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.205194 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.205197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.205199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.205427 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.205658 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.205661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.205662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.205752 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.206093 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.206096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.206168 (+ 72us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.206240 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.206340 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":85,"threads_started":1}
I20260620 07:07:19.206558 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.199898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40051 (local address 127.8.24.62:43017)
0620 07:07:19.200011 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.200017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.200200 (+ 183us) server_negotiation.cc:407] Connection header received
0620 07:07:19.200286 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.200289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.200336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.200421 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.200772 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.201550 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.202422 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.202645 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.202716 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.203811 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.203815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.203817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.203840 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.205095 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.205515 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.205517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.205518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.205590 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.205820 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.205822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.205823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.206009 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.206092 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.206295 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.206406 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
I20260620 07:07:19.216362 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43017
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:19.323848 16024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:19.324100 16024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:19.324148 16024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:19.328075 16024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:19.328130 16024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:19.328166 16024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:07:19.328207 16024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:19.333592 16024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:19.334661 16024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:19.335709 16024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:19.344653 16030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:19.344663 16029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.344753 16024 server_base.cc:1034] running on GCE node
W20260620 07:07:19.344677 16032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.345263 16024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:19.345808 16024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:19.346956 16024 hybrid_clock.cc:648] HybridClock initialized: now 1781939239346935 us; error 35 us; skew 500 ppm
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:19.349382 16024 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260620 07:07:19.350168 16024 webserver.cc:460] Webserver started at http://127.8.24.2:39959/ using document root <none> and password file <none>
I20260620 07:07:19.350639 16024 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:19.350693 16024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:19.350842 16024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:19.352782 16024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "1a2ed005ebf54d889120b7c05e385d7d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "6254b4077f479d52582ff090c580736d"
server_key_iv: "867035e85127deb4b1fef05ace3c4752"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.353273 16024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "1a2ed005ebf54d889120b7c05e385d7d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "6254b4077f479d52582ff090c580736d"
server_key_iv: "867035e85127deb4b1fef05ace3c4752"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.356724 16024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:19.358945 16039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.360139 16024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:19.360231 16024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "1a2ed005ebf54d889120b7c05e385d7d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "6254b4077f479d52582ff090c580736d"
server_key_iv: "867035e85127deb4b1fef05ace3c4752"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.360327 16024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:19.383744 16024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:19.386445 16024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:19.386622 16024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:19.388113 16024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:19.388163 16024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.388216 16024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:19.388245 16024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:19.398630 16024 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39007
I20260620 07:07:19.398674 16154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39007 every 8 connection(s)
I20260620 07:07:19.399674 16024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:19.401299 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.388780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51477 (local address 127.8.24.62:43017)
0620 07:07:19.388939 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.388945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.389896 (+ 951us) server_negotiation.cc:407] Connection header received
0620 07:07:19.391104 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.391108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.391170 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.391230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.393054 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.393857 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.395231 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.395473 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.395566 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.397893 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.397898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.397901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.397931 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.399457 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.399923 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.399927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.399929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.400027 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.400372 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.400375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.400377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.400593 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.400676 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.401085 (+ 409us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.401184 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 07:07:19.401786 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16024
I20260620 07:07:19.401901 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:19.402143 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.389214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.2:51477)
0620 07:07:19.389708 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.389765 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.390838 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.391692 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.391702 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.392180 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.392862 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.392894 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.394002 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.394006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.395121 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.395131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.395610 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.395614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.395810 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.396345 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.396361 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.397721 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.399546 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.399551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.399562 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.399834 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.400127 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.400130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.400132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.400290 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.400682 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.400687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.400889 (+ 202us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.401036 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.401261 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":104,"threads_started":1}
I20260620 07:07:19.402165 8288 external_mini_cluster.cc:1428] Setting key 487e9e2d556db7787205dabaefaa5947
I20260620 07:07:19.403316 16155 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43017
I20260620 07:07:19.403488 16155 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:19.403949 16155 heartbeater.cc:511] Master 127.8.24.62:43017 requested a full tablet report, sending...
I20260620 07:07:19.404901 15812 ts_manager.cc:194] Registered new tserver with Master: 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007)
I20260620 07:07:19.405596 15812 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:51477
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:07:19.412717 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.405589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:38613 (local address 127.8.24.62:43017)
0620 07:07:19.405694 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.405699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.406655 (+ 956us) server_negotiation.cc:407] Connection header received
0620 07:07:19.406743 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.406746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.406784 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.406844 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.407268 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.407794 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.408661 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.408863 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.408944 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.409906 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.409909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.409911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.409931 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.411200 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.411617 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.411621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.411622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.411691 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.411917 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.411921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.411922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.412107 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.412190 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.412418 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.412524 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:19.412773 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.405496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.2:38613)
0620 07:07:19.406572 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.406584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.406673 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.406867 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.406870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.407063 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.407166 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.407173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.407904 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.407907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.408561 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.408570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.408959 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.408963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.409045 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.409476 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.409487 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.409785 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.411291 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.411295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.411297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.411527 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.411765 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.411768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.411770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.411845 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.412194 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.412197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.412279 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.412359 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.412480 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260620 07:07:19.425846 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:43017
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:19.531450 16164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:19.531700 16164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:19.531741 16164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:19.535728 16164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:19.535781 16164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:19.535825 16164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 07:07:19.535877 16164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:19.541554 16164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.24.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:19.542565 16164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:19.543617 16164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:19.552524 16170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:19.552577 16169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.552585 16164 server_base.cc:1034] running on GCE node
W20260620 07:07:19.552538 16172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:19.553099 16164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:19.553716 16164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:19.554873 16164 hybrid_clock.cc:648] HybridClock initialized: now 1781939239554855 us; error 28 us; skew 500 ppm
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:19.557485 16164 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260620 07:07:19.558266 16164 webserver.cc:460] Webserver started at http://127.8.24.3:45975/ using document root <none> and password file <none>
I20260620 07:07:19.558779 16164 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:19.558840 16164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:19.559017 16164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:19.561087 16164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "948c2ebc37fe408795022c3f1ebab86d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "fd116f5206e329c142ed01c9df3bdee2"
server_key_iv: "fb6a7e237dcd404e29e35452afe829f1"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.561664 16164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "948c2ebc37fe408795022c3f1ebab86d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "fd116f5206e329c142ed01c9df3bdee2"
server_key_iv: "fb6a7e237dcd404e29e35452afe829f1"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.565222 16164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:19.567348 16179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.568485 16164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:19.568579 16164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "948c2ebc37fe408795022c3f1ebab86d"
format_stamp: "Formatted at 2026-06-20 07:07:19 on dist-test-slave-wjw3"
server_key: "fd116f5206e329c142ed01c9df3bdee2"
server_key_iv: "fb6a7e237dcd404e29e35452afe829f1"
server_key_version: "encryptionkey@0"
I20260620 07:07:19.568687 16164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:19.584842 16164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:19.587296 16164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:19.587491 16164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:19.588793 16164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:19.588837 16164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:19.588886 16164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:19.588914 16164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:19.598702 16164 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40261
I20260620 07:07:19.598776 16294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40261 every 8 connection(s)
I20260620 07:07:19.599587 16164 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:19.601289 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16164
I20260620 07:07:19.601380 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:19.601517 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.589519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:55423 (local address 127.8.24.62:43017)
0620 07:07:19.589635 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.589639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.590625 (+ 986us) server_negotiation.cc:407] Connection header received
0620 07:07:19.591617 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.591622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.591686 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.591768 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.592900 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.593419 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.594581 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.594826 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.594919 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.598065 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.598068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.598070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.598097 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.599516 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.600025 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.600028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.600030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.600127 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.600480 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.600484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.600485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.600717 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.600808 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.601241 (+ 433us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.601357 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:19.601624 8288 external_mini_cluster.cc:1428] Setting key d73b45782cc903eb68c72be3f511f4c8
I20260620 07:07:19.602308 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.590034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.3:55423)
0620 07:07:19.590481 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.590519 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.591346 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.592001 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.592008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.592314 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.592752 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.592764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.593531 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.593535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.594464 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.594472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.594970 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.594975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.595142 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.595744 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.595759 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.597892 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.599603 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.599608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.599623 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.599919 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.600233 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.600237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.600238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.600391 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.600800 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.600805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.601022 (+ 217us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.601182 (+ 160us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.601468 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260620 07:07:19.603526 16295 heartbeater.cc:347] Connected to a master server at 127.8.24.62:43017
I20260620 07:07:19.603680 16295 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:19.604221 16295 heartbeater.cc:511] Master 127.8.24.62:43017 requested a full tablet report, sending...
I20260620 07:07:19.605172 15810 ts_manager.cc:194] Registered new tserver with Master: 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:19.605927 15810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:55423
I20260620 07:07:19.611646 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.605310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:56749 (local address 127.8.24.62:43017)
0620 07:07:19.605404 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.605407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.605605 (+ 198us) server_negotiation.cc:407] Connection header received
0620 07:07:19.605714 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.605716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.605748 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.605807 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.606445 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.606938 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.607932 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.608134 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.608202 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.609013 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.609016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.609017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.609037 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.610197 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.610575 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.610579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.610581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.610664 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.610901 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.610905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.610907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.611078 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.611158 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.611382 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.611503 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 07:07:19.611670 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.605241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.8.24.3:56749)
0620 07:07:19.605542 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.605555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.605643 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.605830 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.605833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.606011 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:19.606116 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.606123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.607051 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.607054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.607847 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.607856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.608207 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.608209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.608271 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.608627 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:19.608637 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:19.608908 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:19.610289 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.610292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.610294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.610493 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.610747 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:19.610750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:19.610752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:19.610826 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:19.611160 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:19.611163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:19.611243 (+ 80us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.611321 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.611494 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260620 07:07:19.614935 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 07:07:19.635885 15861 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:19.630496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.8.24.62:43017)
0620 07:07:19.630604 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.630608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.630618 (+ 10us) server_negotiation.cc:407] Connection header received
0620 07:07:19.630653 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.630655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.630689 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.630746 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.631361 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.631916 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.632687 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.632886 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.632959 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.635737 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33760: BlockingRecv error: recv got EOF from 127.0.0.1:33760 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 07:07:19 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939239, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:19.644394 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.637281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.8.24.62:43017)
0620 07:07:19.637397 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.637401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.637547 (+ 146us) server_negotiation.cc:407] Connection header received
0620 07:07:19.637640 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.637642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.637677 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.637737 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:19.638243 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.638747 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.639524 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.639731 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.639801 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.641865 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:19.641868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:19.641870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:19.641892 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:19.643094 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.643494 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.643497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.643498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.643577 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:19.643771 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:19.643773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:19.643774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:19.643924 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:19.644015 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.644156 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.644240 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:07:19.646871 15810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:19.649782 15810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:19.663794 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.657605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.8.24.2:39007)
0620 07:07:19.657912 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.657924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.658688 (+ 764us) server_negotiation.cc:407] Connection header received
0620 07:07:19.658916 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.658922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.659141 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.659266 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:19.659827 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.660788 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.662525 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.663256 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.663346 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.663504 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.663553 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.663649 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
I20260620 07:07:19.663803 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.657557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40261 (local address 127.0.0.1:53816)
0620 07:07:19.658932 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.658956 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.659029 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.659917 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.659919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.659935 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:19.660008 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.660013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.661245 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.661249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.662287 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.662296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.663252 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.663255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.663344 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.663387 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.663515 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.663655 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"mutex_wait_us":62,"thread_start_us":83,"threads_started":1}
I20260620 07:07:19.663858 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.659010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.8.24.3:40261)
0620 07:07:19.659503 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.659514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.659535 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:07:19.659599 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.659604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.659780 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.659954 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:19.660090 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.661147 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.662393 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.663149 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.663256 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.663422 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.663649 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.663738 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":102,"threads_started":1}
I20260620 07:07:19.664286 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.657460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39007 (local address 127.0.0.1:46792)
0620 07:07:19.658564 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.658613 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.658736 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.659517 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.659523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.659546 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:19.659689 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.659695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.660891 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.660895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.662414 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.662424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.663358 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.663361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.663456 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.663476 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.663541 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.664159 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":57,"thread_start_us":65,"threads_started":1}
I20260620 07:07:19.665082 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.658015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.0.0.1:44918)
0620 07:07:19.659058 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.659082 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.659179 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.659663 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.659665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.659682 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:19.659782 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.659788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.661148 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.661151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.662414 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.662424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.664673 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.664677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.664809 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.664834 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.664891 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.664966 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":86,"thread_start_us":86,"threads_started":1}
I20260620 07:07:19.665511 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.658190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.8.24.1:44527)
0620 07:07:19.658606 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.658618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.659270 (+ 652us) server_negotiation.cc:407] Connection header received
0620 07:07:19.659348 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.659354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.659543 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.659673 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:19.659886 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.661042 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.663518 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.664328 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.664422 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.665256 (+ 834us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.665304 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.665383 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":111,"threads_started":1}
I20260620 07:07:19.666880 16088 tablet_service.cc:1467] Processing CreateTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 (DEFAULT_TABLE table=test-table [id=576e11e02aa64695acaec143979c6b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.666802 16229 tablet_service.cc:1467] Processing CreateTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 (DEFAULT_TABLE table=test-table [id=576e11e02aa64695acaec143979c6b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.667769 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8834c3661a0e410d888f5aa8bc3b7937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.667810 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8834c3661a0e410d888f5aa8bc3b7937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.668177 15949 tablet_service.cc:1467] Processing CreateTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 (DEFAULT_TABLE table=test-table [id=576e11e02aa64695acaec143979c6b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.669116 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8834c3661a0e410d888f5aa8bc3b7937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.674969 16324 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Bootstrap starting.
I20260620 07:07:19.675756 16323 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Bootstrap starting.
I20260620 07:07:19.676340 16325 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Bootstrap starting.
I20260620 07:07:19.677189 16324 tablet_bootstrap.cc:654] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.677750 16323 tablet_bootstrap.cc:654] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.678056 16325 tablet_bootstrap.cc:654] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.678063 16324 log.cc:824] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:19.678508 16323 log.cc:824] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:19.678755 16325 log.cc:824] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:19.680054 16324 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: No bootstrap required, opened a new log
I20260620 07:07:19.680280 16324 ts_tablet_manager.cc:1397] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:07:19.680362 16323 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: No bootstrap required, opened a new log
I20260620 07:07:19.680526 16323 ts_tablet_manager.cc:1397] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:07:19.680619 16325 tablet_bootstrap.cc:492] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: No bootstrap required, opened a new log
I20260620 07:07:19.680778 16325 ts_tablet_manager.cc:1397] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:07:19.683301 16323 raft_consensus.cc:348] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.683528 16323 raft_consensus.cc:374] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.683610 16323 raft_consensus.cc:729] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.684001 16323 consensus_queue.cc:260] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.684087 16324 raft_consensus.cc:348] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.684218 16325 raft_consensus.cc:348] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.684381 16324 raft_consensus.cc:374] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.684466 16324 raft_consensus.cc:729] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.684571 16325 raft_consensus.cc:374] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.684578 16155 heartbeater.cc:503] Master 127.8.24.62:43017 was elected leader, sending a full tablet report...
I20260620 07:07:19.684629 16325 raft_consensus.cc:729] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32623289074e4faeb18eef1829883885, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.684916 16324 consensus_queue.cc:260] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.684981 16325 consensus_queue.cc:260] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.685524 16015 heartbeater.cc:503] Master 127.8.24.62:43017 was elected leader, sending a full tablet report...
I20260620 07:07:19.685875 16325 ts_tablet_manager.cc:1428] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:07:19.686445 16295 heartbeater.cc:503] Master 127.8.24.62:43017 was elected leader, sending a full tablet report...
I20260620 07:07:19.687275 16323 ts_tablet_manager.cc:1428] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260620 07:07:19.687542 16324 ts_tablet_manager.cc:1428] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260620 07:07:19.691735 16016 tablet.cc:2369] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:19.784327 16329 raft_consensus.cc:482] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:19.784538 16329 raft_consensus.cc:504] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.785586 16329 leader_election.cc:288] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261), 32623289074e4faeb18eef1829883885 (127.8.24.1:44527)
I20260620 07:07:19.789116 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.785818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.8.24.2:52327)
0620 07:07:19.785969 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.785984 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.786082 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.786339 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.786342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.786359 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:19.786486 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.786491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.787204 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.787206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.788206 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.788216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.788809 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.788811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.788885 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.788905 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.788958 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.789012 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 07:07:19.789184 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.785913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52327 (local address 127.8.24.1:44527)
0620 07:07:19.786033 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.786038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.786055 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:07:19.786190 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.786194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.786242 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.786303 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:19.786580 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.787106 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.788320 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.788703 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.788798 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.788863 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.788985 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.789051 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:19.789695 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8834c3661a0e410d888f5aa8bc3b7937" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885" is_pre_election: true
I20260620 07:07:19.789779 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.785928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40255 (local address 127.8.24.3:40261)
0620 07:07:19.786051 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.786056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.786369 (+ 313us) server_negotiation.cc:407] Connection header received
0620 07:07:19.786445 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.786447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.786485 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.786541 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:19.787113 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.787661 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.788775 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.789280 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.789371 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.789425 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.789614 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.789681 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 07:07:19.789779 16332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.785818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40261 (local address 127.8.24.2:40255)
0620 07:07:19.786274 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:19.786285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:19.786376 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:19.786569 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:19.786573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:19.786589 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:19.787007 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.787013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.787769 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.787772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.788668 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:19.788675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.789393 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.789397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:19.789496 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.789514 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:07:19.789575 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:07:19.789636 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":122,"spinlock_wait_cycles":768,"thread_start_us":127,"threads_started":1}
I20260620 07:07:19.789994 15969 raft_consensus.cc:2454] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:19.790285 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8834c3661a0e410d888f5aa8bc3b7937" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d" is_pre_election: true
I20260620 07:07:19.790480 16041 leader_election.cc:302] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885; no voters:
I20260620 07:07:19.790566 16249 raft_consensus.cc:2454] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:19.790747 16329 raft_consensus.cc:2781] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:19.790840 16329 raft_consensus.cc:482] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:19.790899 16329 raft_consensus.cc:3037] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:19.791890 16329 raft_consensus.cc:504] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.792199 16329 leader_election.cc:288] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Requested vote from peers 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261), 32623289074e4faeb18eef1829883885 (127.8.24.1:44527)
I20260620 07:07:19.792512 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8834c3661a0e410d888f5aa8bc3b7937" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d"
I20260620 07:07:19.792518 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8834c3661a0e410d888f5aa8bc3b7937" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885"
I20260620 07:07:19.792682 16249 raft_consensus.cc:3037] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:19.792685 15969 raft_consensus.cc:3037] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:19.793967 15969 raft_consensus.cc:2454] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:19.794175 16249 raft_consensus.cc:2454] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:19.794255 16041 leader_election.cc:302] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885; no voters:
I20260620 07:07:19.794523 16329 raft_consensus.cc:2781] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:19.794726 16329 raft_consensus.cc:686] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Becoming Leader. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Running, Role: LEADER
I20260620 07:07:19.795030 16329 consensus_queue.cc:237] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } }
I20260620 07:07:19.797986 15812 catalog_manager.cc:5495] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2). New cstate: current_term: 1 leader_uuid: "1a2ed005ebf54d889120b7c05e385d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:19.812348 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:19.809420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.8.24.2:39007)
0620 07:07:19.809523 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:19.809526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:19.809539 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:19.809663 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:19.809665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:19.809713 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:19.809792 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:19.809959 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.810435 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.811174 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:19.811396 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:19.811501 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:19.811696 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:19.812087 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:19.812164 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:19.812204 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 07:07:19.812250 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:07:19.817699 16249 raft_consensus.cc:1261] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:19.817700 15969 raft_consensus.cc:1261] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:19.818367 16333 consensus_queue.cc:1035] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:19.818538 16329 consensus_queue.cc:1035] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:19.832621 16337 mvcc.cc:205] Tried to move back new op lower bound from 7298823126286811136 to 7298823126203113472. Current Snapshot: MvccSnapshot[applied={T|T < 7298823126286811136 or (T in {7298823126286811136})}]
I20260620 07:07:19.833469 15812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:19.833609 15812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:19.835758 15812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Sending DeleteTablet for 3 replicas of tablet 8834c3661a0e410d888f5aa8bc3b7937
I20260620 07:07:19.836486 16088 tablet_service.cc:1514] Processing DeleteTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:19 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46792
I20260620 07:07:19.836512 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:19 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53816
I20260620 07:07:19.837149 16347 tablet_replica.cc:330] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: stopping tablet replica
I20260620 07:07:19.837162 15949 tablet_service.cc:1514] Processing DeleteTablet for tablet 8834c3661a0e410d888f5aa8bc3b7937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:19 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44918
I20260620 07:07:19.837466 16347 raft_consensus.cc:2229] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:19.837733 16348 tablet_replica.cc:330] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: stopping tablet replica
I20260620 07:07:19.837775 16347 raft_consensus.cc:2258] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:19.837723 15812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:19.838085 15812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:19.838109 16346 tablet_replica.cc:330] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: stopping tablet replica
I20260620 07:07:19.838199 16348 raft_consensus.cc:2229] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:19.838366 16346 raft_consensus.cc:2229] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:19.838470 16348 raft_consensus.cc:2258] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:19.838617 16346 raft_consensus.cc:2258] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:19.839923 16348 ts_tablet_manager.cc:1905] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:19.841372 16347 ts_tablet_manager.cc:1905] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:19.842790 15949 tablet_service.cc:1467] Processing CreateTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 (DEFAULT_TABLE table=test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.843041 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bbbf8eed87143aa962849d04acf80a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.843081 16229 tablet_service.cc:1467] Processing CreateTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 (DEFAULT_TABLE table=test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.843315 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bbbf8eed87143aa962849d04acf80a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.843884 16347 ts_tablet_manager.cc:1918] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:19.843961 16347 log.cc:1196] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/8834c3661a0e410d888f5aa8bc3b7937
I20260620 07:07:19.844231 16347 ts_tablet_manager.cc:1939] T 8834c3661a0e410d888f5aa8bc3b7937 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting consensus metadata
I20260620 07:07:19.845655 16324 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Bootstrap starting.
I20260620 07:07:19.845655 16325 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Bootstrap starting.
I20260620 07:07:19.846413 16346 ts_tablet_manager.cc:1905] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:19.846544 16325 tablet_bootstrap.cc:654] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.846544 16324 tablet_bootstrap.cc:654] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.847311 16088 tablet_service.cc:1467] Processing CreateTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 (DEFAULT_TABLE table=test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:19.847553 15796 catalog_manager.cc:4841] TS 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007): tablet 8834c3661a0e410d888f5aa8bc3b7937 (table test-table [id=576e11e02aa64695acaec143979c6b48]) successfully deleted
I20260620 07:07:19.847826 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bbbf8eed87143aa962849d04acf80a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:19.848594 16346 ts_tablet_manager.cc:1918] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:19.848668 16346 log.cc:1196] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/8834c3661a0e410d888f5aa8bc3b7937
I20260620 07:07:19.848922 16346 ts_tablet_manager.cc:1939] T 8834c3661a0e410d888f5aa8bc3b7937 P 948c2ebc37fe408795022c3f1ebab86d: Deleting consensus metadata
I20260620 07:07:19.849009 16348 ts_tablet_manager.cc:1918] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:19.849085 16348 log.cc:1196] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/8834c3661a0e410d888f5aa8bc3b7937
I20260620 07:07:19.849352 16348 ts_tablet_manager.cc:1939] T 8834c3661a0e410d888f5aa8bc3b7937 P 32623289074e4faeb18eef1829883885: Deleting consensus metadata
I20260620 07:07:19.849892 16323 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Bootstrap starting.
I20260620 07:07:19.850728 16323 tablet_bootstrap.cc:654] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:19.850900 15797 catalog_manager.cc:4841] TS 32623289074e4faeb18eef1829883885 (127.8.24.1:44527): tablet 8834c3661a0e410d888f5aa8bc3b7937 (table test-table [id=576e11e02aa64695acaec143979c6b48]) successfully deleted
I20260620 07:07:19.851409 16325 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: No bootstrap required, opened a new log
I20260620 07:07:19.851472 15799 catalog_manager.cc:4841] TS 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261): tablet 8834c3661a0e410d888f5aa8bc3b7937 (table test-table [id=576e11e02aa64695acaec143979c6b48]) successfully deleted
I20260620 07:07:19.851514 16325 ts_tablet_manager.cc:1397] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260620 07:07:19.851639 16324 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: No bootstrap required, opened a new log
I20260620 07:07:19.851697 16324 ts_tablet_manager.cc:1397] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260620 07:07:19.851913 16325 raft_consensus.cc:348] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.852013 16325 raft_consensus.cc:374] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.852043 16325 raft_consensus.cc:729] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32623289074e4faeb18eef1829883885, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.852073 16324 raft_consensus.cc:348] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.852152 16324 raft_consensus.cc:374] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.852181 16324 raft_consensus.cc:729] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.852154 16325 consensus_queue.cc:260] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.852212 16323 tablet_bootstrap.cc:492] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: No bootstrap required, opened a new log
I20260620 07:07:19.852278 16323 ts_tablet_manager.cc:1397] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:19.852285 16324 consensus_queue.cc:260] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.852383 16325 ts_tablet_manager.cc:1428] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 07:07:19.852499 16324 ts_tablet_manager.cc:1428] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:19.852672 16323 raft_consensus.cc:348] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.852761 16323 raft_consensus.cc:374] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:19.852787 16323 raft_consensus.cc:729] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:19.852890 16323 consensus_queue.cc:260] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:19.853125 16323 ts_tablet_manager.cc:1428] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260620 07:07:19.901008 16156 tablet.cc:2369] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:20.030565 16343 raft_consensus.cc:482] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:20.030717 16343 raft_consensus.cc:504] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.031149 16343 leader_election.cc:288] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.031666 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bbbf8eed87143aa962849d04acf80a3" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885" is_pre_election: true
I20260620 07:07:20.031666 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bbbf8eed87143aa962849d04acf80a3" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d" is_pre_election: true
I20260620 07:07:20.031821 16249 raft_consensus.cc:2454] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:20.031821 15969 raft_consensus.cc:2454] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:20.032172 16041 leader_election.cc:302] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.032348 16343 raft_consensus.cc:2781] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:20.032408 16343 raft_consensus.cc:482] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:20.032462 16343 raft_consensus.cc:3037] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.033496 16343 raft_consensus.cc:504] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.033795 16343 leader_election.cc:288] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Requested vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.034097 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bbbf8eed87143aa962849d04acf80a3" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d"
I20260620 07:07:20.034118 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bbbf8eed87143aa962849d04acf80a3" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885"
I20260620 07:07:20.034214 16249 raft_consensus.cc:3037] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.034227 15969 raft_consensus.cc:3037] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.035051 15969 raft_consensus.cc:2454] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:20.035063 16249 raft_consensus.cc:2454] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:20.035303 16041 leader_election.cc:302] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885; no voters:
I20260620 07:07:20.035480 16343 raft_consensus.cc:2781] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:20.035574 16343 raft_consensus.cc:686] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Becoming Leader. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Running, Role: LEADER
I20260620 07:07:20.035732 16343 consensus_queue.cc:237] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.037163 15811 catalog_manager.cc:5495] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2). New cstate: current_term: 1 leader_uuid: "1a2ed005ebf54d889120b7c05e385d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } health_report { overall_health: UNKNOWN } } }
W20260620 07:07:20.101118 16296 tablet.cc:2369] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 07:07:20 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939238, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.62@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:20.108256 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.100934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:43017 (local address 127.0.0.1:33774)
0620 07:07:20.101187 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.101201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.101310 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.101527 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.101530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.101931 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:20.102024 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.102038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.102698 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.102700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.103260 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.103266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.103677 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.103681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.103755 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.104250 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:20.104261 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:20.105608 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:20.107024 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:20.107030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:20.107035 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:20.107232 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:20.107420 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:20.107422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:20.107423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:20.107521 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:20.107813 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:20.107817 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:20.107913 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.107998 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.108106 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260620 07:07:20.108282 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.100969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.8.24.62:43017)
0620 07:07:20.101095 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.101099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.101273 (+ 174us) server_negotiation.cc:407] Connection header received
0620 07:07:20.101387 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.101389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.101429 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.101490 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:20.102115 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.102620 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.103343 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.103595 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.103667 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.105718 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:20.105721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:20.105723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:20.105742 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:20.106950 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:20.107297 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:20.107300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:20.107301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:20.107363 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:20.107588 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:20.107591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:20.107592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:20.107741 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:20.107810 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.108036 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.108158 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":48}
I20260620 07:07:20.111691 15811 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:33774:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 07:07:20.112388 15811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:20.116595 16088 tablet_service.cc:1467] Processing CreateTablet for tablet 01cb662de7794f71a55ab2e36826b008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0470b7b60f421494b327353d57444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:07:20.116605 15949 tablet_service.cc:1467] Processing CreateTablet for tablet 01cb662de7794f71a55ab2e36826b008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0470b7b60f421494b327353d57444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:07:20.116708 16229 tablet_service.cc:1467] Processing CreateTablet for tablet 01cb662de7794f71a55ab2e36826b008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9b0470b7b60f421494b327353d57444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 07:07:20.116940 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cb662de7794f71a55ab2e36826b008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.116969 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cb662de7794f71a55ab2e36826b008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.117003 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01cb662de7794f71a55ab2e36826b008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.119164 16323 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: Bootstrap starting.
I20260620 07:07:20.119645 16325 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: Bootstrap starting.
I20260620 07:07:20.119858 16324 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: Bootstrap starting.
I20260620 07:07:20.119969 16323 tablet_bootstrap.cc:654] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.120532 16325 tablet_bootstrap.cc:654] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.120661 16324 tablet_bootstrap.cc:654] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.121798 16325 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: No bootstrap required, opened a new log
I20260620 07:07:20.121814 16324 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: No bootstrap required, opened a new log
I20260620 07:07:20.121881 16324 ts_tablet_manager.cc:1397] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.121883 16325 ts_tablet_manager.cc:1397] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 07:07:20.122001 16323 tablet_bootstrap.cc:492] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: No bootstrap required, opened a new log
I20260620 07:07:20.122064 16323 ts_tablet_manager.cc:1397] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 07:07:20.122262 16324 raft_consensus.cc:348] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122354 16324 raft_consensus.cc:374] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.122375 16324 raft_consensus.cc:729] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.122318 16325 raft_consensus.cc:348] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122418 16325 raft_consensus.cc:374] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.122447 16325 raft_consensus.cc:729] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32623289074e4faeb18eef1829883885, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.122396 16323 raft_consensus.cc:348] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122488 16323 raft_consensus.cc:374] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.122515 16323 raft_consensus.cc:729] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.122483 16324 consensus_queue.cc:260] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122563 16325 consensus_queue.cc:260] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122628 16323 consensus_queue.cc:260] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.122723 16324 ts_tablet_manager.cc:1428] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.122678 16330 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.122800 16330 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.122800 16325 ts_tablet_manager.cc:1428] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 07:07:20.122848 16323 ts_tablet_manager.cc:1428] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.122990 16343 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.123005 16331 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.123111 16343 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.123116 16331 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.173039 16331 raft_consensus.cc:482] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:20.173183 16331 raft_consensus.cc:504] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.174254 16331 leader_election.cc:288] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.178102 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.174646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48947 (local address 127.8.24.3:40261)
0620 07:07:20.174749 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.174753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.174931 (+ 178us) server_negotiation.cc:407] Connection header received
0620 07:07:20.175094 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.175098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.175142 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.175202 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:20.175515 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.176014 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177197 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.177581 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177668 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.177733 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.177932 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.177998 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:20.178094 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.174565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50131 (local address 127.8.24.2:39007)
0620 07:07:20.174693 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.174698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.174990 (+ 292us) server_negotiation.cc:407] Connection header received
0620 07:07:20.175103 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.175105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.175142 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.175203 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:20.175518 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.176011 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177197 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.177580 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177666 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.177733 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.177901 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.177969 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:20.178121 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.174515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39007 (local address 127.8.24.1:50131)
0620 07:07:20.174902 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.174928 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.175017 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.175220 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.175222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.175254 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:20.175396 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.175401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.176108 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.176112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.177086 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.177095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.177682 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.177796 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.177815 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.177870 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.177946 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":96,"spinlock_wait_cycles":13056,"thread_start_us":80,"threads_started":1}
I20260620 07:07:20.178122 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.174515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40261 (local address 127.8.24.1:48947)
0620 07:07:20.174844 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.174855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.174964 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.175221 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.175224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.175260 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:20.175388 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.175393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.176114 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.176117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.177077 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.177086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.177682 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.177686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.177820 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.177837 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.177904 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.177969 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":85,"spinlock_wait_cycles":3968,"thread_start_us":109,"threads_started":1}
I20260620 07:07:20.178555 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cb662de7794f71a55ab2e36826b008" candidate_uuid: "32623289074e4faeb18eef1829883885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2ed005ebf54d889120b7c05e385d7d" is_pre_election: true
I20260620 07:07:20.178785 16109 raft_consensus.cc:2454] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32623289074e4faeb18eef1829883885 in term 0.
I20260620 07:07:20.179186 15900 leader_election.cc:302] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885; no voters:
I20260620 07:07:20.179219 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cb662de7794f71a55ab2e36826b008" candidate_uuid: "32623289074e4faeb18eef1829883885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d" is_pre_election: true
I20260620 07:07:20.179342 16249 raft_consensus.cc:2454] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32623289074e4faeb18eef1829883885 in term 0.
I20260620 07:07:20.179474 16331 raft_consensus.cc:2781] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:20.179535 16331 raft_consensus.cc:482] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:20.179580 16331 raft_consensus.cc:3037] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.180500 16331 raft_consensus.cc:504] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.180814 16331 leader_election.cc:288] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [CANDIDATE]: Term 1 election: Requested vote from peers 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.181137 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cb662de7794f71a55ab2e36826b008" candidate_uuid: "32623289074e4faeb18eef1829883885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d"
I20260620 07:07:20.181226 16249 raft_consensus.cc:3037] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.182057 16249 raft_consensus.cc:2454] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32623289074e4faeb18eef1829883885 in term 1.
I20260620 07:07:20.182269 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01cb662de7794f71a55ab2e36826b008" candidate_uuid: "32623289074e4faeb18eef1829883885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2ed005ebf54d889120b7c05e385d7d"
I20260620 07:07:20.182309 15903 leader_election.cc:302] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32623289074e4faeb18eef1829883885, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.182391 16109 raft_consensus.cc:3037] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.182472 16331 raft_consensus.cc:2781] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:20.182658 16331 raft_consensus.cc:686] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [term 1 LEADER]: Becoming Leader. State: Replica: 32623289074e4faeb18eef1829883885, State: Running, Role: LEADER
I20260620 07:07:20.182917 16331 consensus_queue.cc:237] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.183526 16109 raft_consensus.cc:2454] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32623289074e4faeb18eef1829883885 in term 1.
I20260620 07:07:20.184173 16364 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: TxnStatusTablet state changed. Reason: New leader 32623289074e4faeb18eef1829883885. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.184329 16364 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:07:20.184859 16366 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 07:07:20.184986 16366 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 07:07:20.185199 15812 catalog_manager.cc:5495] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32623289074e4faeb18eef1829883885 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:20.204962 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.200423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.0.0.1:44932)
0620 07:07:20.200645 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.200659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.200762 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.201078 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.201080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.201088 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:07:20.201193 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.201198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.202157 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.202160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.203145 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.203155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.203754 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.203757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.203847 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.203895 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.204668 (+ 773us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.204681 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.204741 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.204824 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260620 07:07:20.205443 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.200517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.8.24.1:44527)
0620 07:07:20.200631 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.200635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.200703 (+ 68us) server_negotiation.cc:407] Connection header received
0620 07:07:20.200897 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.200900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.200961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.201056 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.201277 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.202061 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.203256 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.203634 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.203752 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.204030 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.204564 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.204689 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.205289 (+ 600us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.205343 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:07:20.207794 16249 raft_consensus.cc:1261] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Refusing update from remote peer 32623289074e4faeb18eef1829883885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.208024 16109 raft_consensus.cc:1261] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Refusing update from remote peer 32623289074e4faeb18eef1829883885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.208206 16331 consensus_queue.cc:1035] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.208520 16364 consensus_queue.cc:1035] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.209474 16330 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: TxnStatusTablet state changed. Reason: New leader 32623289074e4faeb18eef1829883885. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.209589 16330 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.211148 16364 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.211244 16364 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:07:20.212015 16330 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.212114 16330 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 948c2ebc37fe408795022c3f1ebab86d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.212085 16338 mvcc.cc:205] Tried to move back new op lower bound from 7298823127886807040 to 7298823127792214016. Current Snapshot: MvccSnapshot[applied={T|T < 7298823127886807040}]
I20260620 07:07:20.212953 16364 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.213034 16364 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 32623289074e4faeb18eef1829883885: This TxnStatusTablet replica's current role is: LEADER
I20260620 07:07:20.211360 16343 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: TxnStatusTablet state changed. Reason: New leader 32623289074e4faeb18eef1829883885. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.214102 16343 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.216773 16343 tablet_replica.cc:439] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32623289074e4faeb18eef1829883885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } } }
I20260620 07:07:20.216864 16343 tablet_replica.cc:442] T 01cb662de7794f71a55ab2e36826b008 P 1a2ed005ebf54d889120b7c05e385d7d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 07:07:20.225333 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.222340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:49111 (local address 127.8.24.1:44527)
0620 07:07:20.222437 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.222442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.222629 (+ 187us) server_negotiation.cc:407] Connection header received
0620 07:07:20.222736 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.222739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.222780 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.222837 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.223056 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.223590 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.224410 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.224624 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.224704 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.224964 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.225026 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.225079 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.225170 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.225215 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 07:07:20.225360 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.222279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.8.24.2:49111)
0620 07:07:20.222557 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.222569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.222661 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.222850 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.222852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.222859 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:07:20.222972 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.222977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.223689 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.223692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.224306 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.224314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.224713 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.224717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.224811 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.224849 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.225100 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.225106 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.225162 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.225243 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260620 07:07:20.231307 15969 raft_consensus.cc:1261] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.231626 16249 raft_consensus.cc:1261] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.231629 16329 consensus_queue.cc:1035] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.231940 16329 consensus_queue.cc:1035] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.238994 16336 mvcc.cc:205] Tried to move back new op lower bound from 7298823127984439296 to 7298823127187599360. Current Snapshot: MvccSnapshot[applied={T|T < 7298823127984439296}]
I20260620 07:07:20.253468 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.249803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:52569 (local address 127.8.24.2:39007)
0620 07:07:20.249913 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.249917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.249954 (+ 37us) server_negotiation.cc:407] Connection header received
0620 07:07:20.249994 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.249997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.250044 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.250112 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.250446 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.251227 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.252478 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.252695 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.252769 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.253066 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.253145 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.253205 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.253296 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.253347 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:07:20.253526 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.249549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39007 (local address 127.8.24.1:52569)
0620 07:07:20.249823 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.249836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.249906 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.250114 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.250116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.250121 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:07:20.250235 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.250348 (+ 113us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.251350 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.251353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.252373 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.252384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.252790 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.252794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.252895 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.252945 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.253223 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.253229 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.253287 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.253395 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260620 07:07:20.275465 15812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.275612 15812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.277410 15812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Sending DeleteTablet for 3 replicas of tablet 9bbbf8eed87143aa962849d04acf80a3
I20260620 07:07:20.277875 15949 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44918
I20260620 07:07:20.277997 16088 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46792
I20260620 07:07:20.277992 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bbbf8eed87143aa962849d04acf80a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53816
I20260620 07:07:20.278048 16348 tablet_replica.cc:330] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: stopping tablet replica
I20260620 07:07:20.278146 16347 tablet_replica.cc:330] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: stopping tablet replica
I20260620 07:07:20.278146 16348 raft_consensus.cc:2229] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.278237 16347 raft_consensus.cc:2229] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:20.278244 16348 raft_consensus.cc:2258] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.278379 16347 raft_consensus.cc:2258] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.278532 16346 tablet_replica.cc:330] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: stopping tablet replica
I20260620 07:07:20.278635 16346 raft_consensus.cc:2229] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.278717 16346 raft_consensus.cc:2258] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.278992 16348 ts_tablet_manager.cc:1905] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.279326 16347 ts_tablet_manager.cc:1905] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.281783 16348 ts_tablet_manager.cc:1918] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.281992 16348 log.cc:1196] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/9bbbf8eed87143aa962849d04acf80a3
I20260620 07:07:20.282004 16347 ts_tablet_manager.cc:1918] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.282183 16347 log.cc:1196] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/9bbbf8eed87143aa962849d04acf80a3
I20260620 07:07:20.282285 16348 ts_tablet_manager.cc:1939] T 9bbbf8eed87143aa962849d04acf80a3 P 32623289074e4faeb18eef1829883885: Deleting consensus metadata
I20260620 07:07:20.282493 16347 ts_tablet_manager.cc:1939] T 9bbbf8eed87143aa962849d04acf80a3 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting consensus metadata
I20260620 07:07:20.282977 15797 catalog_manager.cc:4841] TS 32623289074e4faeb18eef1829883885 (127.8.24.1:44527): tablet 9bbbf8eed87143aa962849d04acf80a3 (table test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]) successfully deleted
I20260620 07:07:20.283118 15796 catalog_manager.cc:4841] TS 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007): tablet 9bbbf8eed87143aa962849d04acf80a3 (table test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]) successfully deleted
I20260620 07:07:20.283360 16346 ts_tablet_manager.cc:1905] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.285283 16346 ts_tablet_manager.cc:1918] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.285346 16346 log.cc:1196] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/9bbbf8eed87143aa962849d04acf80a3
I20260620 07:07:20.285531 16346 ts_tablet_manager.cc:1939] T 9bbbf8eed87143aa962849d04acf80a3 P 948c2ebc37fe408795022c3f1ebab86d: Deleting consensus metadata
I20260620 07:07:20.286080 15799 catalog_manager.cc:4841] TS 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261): tablet 9bbbf8eed87143aa962849d04acf80a3 (table test-table [id=9f77c98d709b4991ab9cf8aa214dde9f]) successfully deleted
Jun 20 07:07:20 dist-test-slave-wjw3 krb5kdc[15767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 07:07:20 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:20 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 07:07:20.308143 15861 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:20.301293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33776 (local address 127.8.24.62:43017)
0620 07:07:20.301436 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.301441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.301602 (+ 161us) server_negotiation.cc:407] Connection header received
0620 07:07:20.301740 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.301743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.301785 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.301861 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:20.302687 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.303244 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.304484 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.304748 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.304826 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.308013 (+ 3187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33776: BlockingRecv error: recv got EOF from 127.0.0.1:33776 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 07:07:20 dist-test-slave-wjw3 krb5kdc[15767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM
I20260620 07:07:20.317512 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.309643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.8.24.62:43017)
0620 07:07:20.309778 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.309782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.309869 (+ 87us) server_negotiation.cc:407] Connection header received
0620 07:07:20.309994 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.309997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.310037 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.310095 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:20.310644 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.311142 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.311929 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.312140 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.312221 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.314714 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:20.314717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:20.314719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:20.314741 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:20.316020 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:20.316461 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:20.316464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:20.316465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:20.316529 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:20.316788 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:20.316790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:20.316791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:20.316953 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:20.317044 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.317302 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.317393 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:07:20.320009 15812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:20.320415 15812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:20.325028 15949 tablet_service.cc:1467] Processing CreateTablet for tablet c2e180c763254d90af1b05e94cee6a42 (DEFAULT_TABLE table=test-table [id=264412f59a8e4fefaa4efd5c46deec82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.325160 16229 tablet_service.cc:1467] Processing CreateTablet for tablet c2e180c763254d90af1b05e94cee6a42 (DEFAULT_TABLE table=test-table [id=264412f59a8e4fefaa4efd5c46deec82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.325327 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e180c763254d90af1b05e94cee6a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.325407 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e180c763254d90af1b05e94cee6a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.327147 16324 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Bootstrap starting.
I20260620 07:07:20.328051 16324 tablet_bootstrap.cc:654] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.328648 16325 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Bootstrap starting.
I20260620 07:07:20.329192 16324 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: No bootstrap required, opened a new log
I20260620 07:07:20.329274 16324 ts_tablet_manager.cc:1397] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.329380 16325 tablet_bootstrap.cc:654] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.329689 16324 raft_consensus.cc:348] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.329784 16324 raft_consensus.cc:374] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.329809 16324 raft_consensus.cc:729] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.329932 16324 consensus_queue.cc:260] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.330188 16324 ts_tablet_manager.cc:1428] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.330691 16088 tablet_service.cc:1467] Processing CreateTablet for tablet c2e180c763254d90af1b05e94cee6a42 (DEFAULT_TABLE table=test-table [id=264412f59a8e4fefaa4efd5c46deec82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.330930 16325 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: No bootstrap required, opened a new log
I20260620 07:07:20.330933 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e180c763254d90af1b05e94cee6a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.330996 16325 ts_tablet_manager.cc:1397] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 07:07:20.331406 16325 raft_consensus.cc:348] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.331532 16325 raft_consensus.cc:374] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.331561 16325 raft_consensus.cc:729] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32623289074e4faeb18eef1829883885, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.331663 16325 consensus_queue.cc:260] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.331904 16325 ts_tablet_manager.cc:1428] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.333009 16323 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Bootstrap starting.
I20260620 07:07:20.333767 16323 tablet_bootstrap.cc:654] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.334762 16323 tablet_bootstrap.cc:492] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: No bootstrap required, opened a new log
I20260620 07:07:20.334825 16323 ts_tablet_manager.cc:1397] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.335139 16323 raft_consensus.cc:348] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.335217 16323 raft_consensus.cc:374] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.335237 16323 raft_consensus.cc:729] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.335347 16323 consensus_queue.cc:260] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.335609 16323 ts_tablet_manager.cc:1428] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.651371 16329 raft_consensus.cc:482] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:20.651526 16329 raft_consensus.cc:504] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.651947 16329 leader_election.cc:288] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.652438 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e180c763254d90af1b05e94cee6a42" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885" is_pre_election: true
I20260620 07:07:20.652436 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e180c763254d90af1b05e94cee6a42" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d" is_pre_election: true
I20260620 07:07:20.652587 16249 raft_consensus.cc:2454] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:20.652585 15969 raft_consensus.cc:2454] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 0.
I20260620 07:07:20.652871 16043 leader_election.cc:302] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 32623289074e4faeb18eef1829883885, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.653049 16329 raft_consensus.cc:2781] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:20.653113 16329 raft_consensus.cc:482] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:20.653142 16329 raft_consensus.cc:3037] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.654066 16329 raft_consensus.cc:504] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.654417 16329 leader_election.cc:288] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Requested vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261)
I20260620 07:07:20.654739 16249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e180c763254d90af1b05e94cee6a42" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948c2ebc37fe408795022c3f1ebab86d"
I20260620 07:07:20.654736 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e180c763254d90af1b05e94cee6a42" candidate_uuid: "1a2ed005ebf54d889120b7c05e385d7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885"
I20260620 07:07:20.654831 16249 raft_consensus.cc:3037] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.654837 15969 raft_consensus.cc:3037] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.655675 16249 raft_consensus.cc:2454] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:20.655764 15969 raft_consensus.cc:2454] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ed005ebf54d889120b7c05e385d7d in term 1.
I20260620 07:07:20.655936 16043 leader_election.cc:302] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ed005ebf54d889120b7c05e385d7d, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.656159 16329 raft_consensus.cc:2781] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:20.656246 16329 raft_consensus.cc:686] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Becoming Leader. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Running, Role: LEADER
I20260620 07:07:20.656378 16329 consensus_queue.cc:237] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.657924 15810 catalog_manager.cc:5495] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2). New cstate: current_term: 1 leader_uuid: "1a2ed005ebf54d889120b7c05e385d7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:20.695535 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.692415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46810 (local address 127.8.24.2:39007)
0620 07:07:20.692585 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.692590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.692603 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:20.692647 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.692649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.692692 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.692750 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.692984 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.693475 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.694414 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.694688 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.694779 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.695082 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.695156 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.695214 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.695307 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.695359 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 07:07:20.697296 15969 raft_consensus.cc:1261] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.697296 16249 raft_consensus.cc:1261] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Refusing update from remote peer 1a2ed005ebf54d889120b7c05e385d7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.697675 16333 consensus_queue.cc:1035] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.697854 16329 consensus_queue.cc:1035] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.703665 15810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.704587 15810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.706591 15810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Sending DeleteTablet for 3 replicas of tablet c2e180c763254d90af1b05e94cee6a42
I20260620 07:07:20.707036 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e180c763254d90af1b05e94cee6a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53816
I20260620 07:07:20.707036 16088 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e180c763254d90af1b05e94cee6a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46792
I20260620 07:07:20.707198 16347 tablet_replica.cc:330] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: stopping tablet replica
I20260620 07:07:20.707212 15949 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e180c763254d90af1b05e94cee6a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44918
I20260620 07:07:20.707319 16347 raft_consensus.cc:2229] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:20.707371 16346 tablet_replica.cc:330] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: stopping tablet replica
I20260620 07:07:20.707458 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e180c763254d90af1b05e94cee6a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53816
I20260620 07:07:20.707502 16346 raft_consensus.cc:2229] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.707530 16347 raft_consensus.cc:2258] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.707607 16346 raft_consensus.cc:2258] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.707742 16348 tablet_replica.cc:330] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: stopping tablet replica
I20260620 07:07:20.707831 16348 raft_consensus.cc:2229] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.707932 16348 raft_consensus.cc:2258] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.708196 16347 ts_tablet_manager.cc:1905] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.708494 15810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 07:07:20.708688 16346 ts_tablet_manager.cc:1905] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.708688 16348 ts_tablet_manager.cc:1905] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 07:07:20.708849 15810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:20.710613 16347 ts_tablet_manager.cc:1918] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:20.710677 16347 log.cc:1196] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/c2e180c763254d90af1b05e94cee6a42
I20260620 07:07:20.710793 16346 ts_tablet_manager.cc:1918] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:20.710793 16348 ts_tablet_manager.cc:1918] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:20.710856 16346 log.cc:1196] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/c2e180c763254d90af1b05e94cee6a42
I20260620 07:07:20.710856 16348 log.cc:1196] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/c2e180c763254d90af1b05e94cee6a42
I20260620 07:07:20.710917 16347 ts_tablet_manager.cc:1939] T c2e180c763254d90af1b05e94cee6a42 P 1a2ed005ebf54d889120b7c05e385d7d: Deleting consensus metadata
I20260620 07:07:20.711095 16346 ts_tablet_manager.cc:1939] T c2e180c763254d90af1b05e94cee6a42 P 948c2ebc37fe408795022c3f1ebab86d: Deleting consensus metadata
I20260620 07:07:20.711107 16348 ts_tablet_manager.cc:1939] T c2e180c763254d90af1b05e94cee6a42 P 32623289074e4faeb18eef1829883885: Deleting consensus metadata
I20260620 07:07:20.711675 15796 catalog_manager.cc:4841] TS 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007): tablet c2e180c763254d90af1b05e94cee6a42 (table test-table [id=264412f59a8e4fefaa4efd5c46deec82]) successfully deleted
I20260620 07:07:20.711702 15797 catalog_manager.cc:4841] TS 32623289074e4faeb18eef1829883885 (127.8.24.1:44527): tablet c2e180c763254d90af1b05e94cee6a42 (table test-table [id=264412f59a8e4fefaa4efd5c46deec82]) successfully deleted
I20260620 07:07:20.711894 15799 catalog_manager.cc:4841] TS 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261): tablet c2e180c763254d90af1b05e94cee6a42 (table test-table [id=264412f59a8e4fefaa4efd5c46deec82]) successfully deleted
W20260620 07:07:20.712141 15799 catalog_manager.cc:4808] TS 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261): delete failed for tablet c2e180c763254d90af1b05e94cee6a42 because the tablet was not found. No further retry: Not found: Tablet not found: c2e180c763254d90af1b05e94cee6a42
I20260620 07:07:20.713500 15949 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac76c4bb6c474084d144013b44fe4a (DEFAULT_TABLE table=test-table [id=7b9a624cdf9c45a18430a4d8f496ab1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.713651 16088 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac76c4bb6c474084d144013b44fe4a (DEFAULT_TABLE table=test-table [id=7b9a624cdf9c45a18430a4d8f496ab1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.713733 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac76c4bb6c474084d144013b44fe4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.713862 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac76c4bb6c474084d144013b44fe4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.713855 16229 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac76c4bb6c474084d144013b44fe4a (DEFAULT_TABLE table=test-table [id=7b9a624cdf9c45a18430a4d8f496ab1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:20.714073 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac76c4bb6c474084d144013b44fe4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:20.715646 16323 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Bootstrap starting.
I20260620 07:07:20.715915 16325 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Bootstrap starting.
I20260620 07:07:20.716112 16324 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Bootstrap starting.
I20260620 07:07:20.716378 16323 tablet_bootstrap.cc:654] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.716617 16325 tablet_bootstrap.cc:654] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.716876 16324 tablet_bootstrap.cc:654] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:20.717414 16323 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: No bootstrap required, opened a new log
I20260620 07:07:20.717501 16323 ts_tablet_manager.cc:1397] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.717500 16325 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: No bootstrap required, opened a new log
I20260620 07:07:20.717574 16325 ts_tablet_manager.cc:1397] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.717860 16323 raft_consensus.cc:348] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.717890 16325 raft_consensus.cc:348] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.717928 16324 tablet_bootstrap.cc:492] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: No bootstrap required, opened a new log
I20260620 07:07:20.717957 16323 raft_consensus.cc:374] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.717967 16325 raft_consensus.cc:374] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.717988 16325 raft_consensus.cc:729] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32623289074e4faeb18eef1829883885, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.717986 16323 raft_consensus.cc:729] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ed005ebf54d889120b7c05e385d7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.718004 16324 ts_tablet_manager.cc:1397] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:20.718072 16325 consensus_queue.cc:260] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.718103 16323 consensus_queue.cc:260] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.718328 16325 ts_tablet_manager.cc:1428] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:07:20.718334 16323 ts_tablet_manager.cc:1428] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:07:20.718389 16324 raft_consensus.cc:348] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.718475 16324 raft_consensus.cc:374] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:20.718503 16324 raft_consensus.cc:729] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Initialized, Role: FOLLOWER
I20260620 07:07:20.718608 16324 consensus_queue.cc:260] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.718842 16324 ts_tablet_manager.cc:1428] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:07:20.882125 16330 raft_consensus.cc:482] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:20.882292 16330 raft_consensus.cc:504] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.883338 16330 leader_election.cc:288] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007)
I20260620 07:07:20.887044 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.883698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:52563 (local address 127.8.24.2:39007)
0620 07:07:20.883833 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.883838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.884139 (+ 301us) server_negotiation.cc:407] Connection header received
0620 07:07:20.884261 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.884264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.884305 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.884364 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:20.884612 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.885095 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886092 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.886474 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886562 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.886621 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.886845 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.886924 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 07:07:20.887060 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.883604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39007 (local address 127.8.24.3:52563)
0620 07:07:20.884046 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.884058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.884163 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.884387 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.884390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.884405 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:20.884516 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.884521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.885189 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.885193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.885975 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.885983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.886575 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.886720 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.886751 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.886817 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.886892 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":54,"thread_start_us":68,"threads_started":1}
I20260620 07:07:20.887049 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.883775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:42035 (local address 127.8.24.1:44527)
0620 07:07:20.883983 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.883987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.884001 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:07:20.884141 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.884144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.884183 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.884255 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:20.884477 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.884979 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886036 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.886451 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886541 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.886609 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.886844 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.886916 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260620 07:07:20.887063 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.883604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.8.24.3:42035)
0620 07:07:20.883926 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.883941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.884037 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.884272 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.884274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.884289 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:20.884383 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.884389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.885100 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.885104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.885916 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.885924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.886559 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.886563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.886712 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.886753 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.886814 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.886892 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":35,"thread_start_us":88,"threads_started":1}
I20260620 07:07:20.887630 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac76c4bb6c474084d144013b44fe4a" candidate_uuid: "948c2ebc37fe408795022c3f1ebab86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885" is_pre_election: true
I20260620 07:07:20.887663 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac76c4bb6c474084d144013b44fe4a" candidate_uuid: "948c2ebc37fe408795022c3f1ebab86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2ed005ebf54d889120b7c05e385d7d" is_pre_election: true
I20260620 07:07:20.887765 15969 raft_consensus.cc:2454] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948c2ebc37fe408795022c3f1ebab86d in term 0.
I20260620 07:07:20.887799 16109 raft_consensus.cc:2454] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948c2ebc37fe408795022c3f1ebab86d in term 0.
I20260620 07:07:20.888121 16181 leader_election.cc:302] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32623289074e4faeb18eef1829883885, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.888363 16330 raft_consensus.cc:2781] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:20.888422 16330 raft_consensus.cc:482] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:20.888458 16330 raft_consensus.cc:3037] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.889447 16330 raft_consensus.cc:504] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.889775 16330 leader_election.cc:288] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [CANDIDATE]: Term 1 election: Requested vote from peers 32623289074e4faeb18eef1829883885 (127.8.24.1:44527), 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007)
I20260620 07:07:20.890094 15969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac76c4bb6c474084d144013b44fe4a" candidate_uuid: "948c2ebc37fe408795022c3f1ebab86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32623289074e4faeb18eef1829883885"
I20260620 07:07:20.890094 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac76c4bb6c474084d144013b44fe4a" candidate_uuid: "948c2ebc37fe408795022c3f1ebab86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2ed005ebf54d889120b7c05e385d7d"
I20260620 07:07:20.890214 16109 raft_consensus.cc:3037] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.890214 15969 raft_consensus.cc:3037] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:20.891258 15969 raft_consensus.cc:2454] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948c2ebc37fe408795022c3f1ebab86d in term 1.
I20260620 07:07:20.891258 16109 raft_consensus.cc:2454] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948c2ebc37fe408795022c3f1ebab86d in term 1.
I20260620 07:07:20.891547 16181 leader_election.cc:302] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32623289074e4faeb18eef1829883885, 948c2ebc37fe408795022c3f1ebab86d; no voters:
I20260620 07:07:20.891702 16330 raft_consensus.cc:2781] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:20.891865 16330 raft_consensus.cc:686] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 1 LEADER]: Becoming Leader. State: Replica: 948c2ebc37fe408795022c3f1ebab86d, State: Running, Role: LEADER
I20260620 07:07:20.892099 16330 consensus_queue.cc:237] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } }
I20260620 07:07:20.894112 15811 catalog_manager.cc:5495] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d reported cstate change: term changed from 0 to 1, leader changed from <none> to 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3). New cstate: current_term: 1 leader_uuid: "948c2ebc37fe408795022c3f1ebab86d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948c2ebc37fe408795022c3f1ebab86d" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40261 } health_report { overall_health: HEALTHY } } }
I20260620 07:07:20.914425 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.911043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.8.24.3:40261)
0620 07:07:20.911303 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.911309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.911324 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:07:20.911428 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.911461 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.911525 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.911627 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.911794 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.912310 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.913050 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.913304 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.913413 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.913585 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.914051 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.914130 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.914181 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.914227 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260620 07:07:20.923265 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.920556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:52569 (local address 127.8.24.1:44527)
0620 07:07:20.920705 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.920708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.920806 (+ 98us) server_negotiation.cc:407] Connection header received
0620 07:07:20.920908 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.920910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.920966 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.921027 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.921183 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.921663 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.922413 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.922609 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.922714 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.922943 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.923006 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.923057 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.923122 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.923165 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:07:20.923283 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.920477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.8.24.3:52569)
0620 07:07:20.920740 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.920752 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.920834 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.921035 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.921036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.921041 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:07:20.921127 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.921130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.921737 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.921739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.922340 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.922346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.922706 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.922710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.922788 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.922827 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.923064 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.923068 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.923107 (+ 39us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.923165 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
I20260620 07:07:20.928692 15969 raft_consensus.cc:1261] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Refusing update from remote peer 948c2ebc37fe408795022c3f1ebab86d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.929029 16109 raft_consensus.cc:1261] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Refusing update from remote peer 948c2ebc37fe408795022c3f1ebab86d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:20.929143 16330 consensus_queue.cc:1035] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32623289074e4faeb18eef1829883885" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.929417 16405 consensus_queue.cc:1035] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2ed005ebf54d889120b7c05e385d7d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:20.946709 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.943014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40261 (local address 127.8.24.1:38981)
0620 07:07:20.943523 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.943536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:20.943636 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:20.943838 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:20.943841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:20.943848 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 07:07:20.943946 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.943952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.944627 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.944630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.945424 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:20.945433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.946154 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.946157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:20.946221 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.946248 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.946485 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.946488 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:07:20.946522 (+ 34us) client_negotiation.cc:240] Negotiation successful
0620 07:07:20.946596 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":93,"threads_started":1}
I20260620 07:07:20.946717 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.943053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38981 (local address 127.8.24.3:40261)
0620 07:07:20.943150 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:20.943154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:20.943609 (+ 455us) server_negotiation.cc:407] Connection header received
0620 07:07:20.943719 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:20.943722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:20.943766 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:20.943824 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:20.944033 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.944530 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.945716 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:20.946057 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:20.946144 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:20.946339 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:20.946423 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:20.946485 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:20.946544 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 07:07:20.946595 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 07:07:20.969533 15809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.969724 15809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:20.971863 15809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271425da39a043d0bd969564f273bd3b: Sending DeleteTablet for 3 replicas of tablet 5eac76c4bb6c474084d144013b44fe4a
I20260620 07:07:20.972378 15949 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac76c4bb6c474084d144013b44fe4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44918
I20260620 07:07:20.972471 16088 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac76c4bb6c474084d144013b44fe4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46792
I20260620 07:07:20.972566 16348 tablet_replica.cc:330] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: stopping tablet replica
I20260620 07:07:20.972662 16347 tablet_replica.cc:330] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: stopping tablet replica
I20260620 07:07:20.972682 16348 raft_consensus.cc:2229] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.972667 16229 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac76c4bb6c474084d144013b44fe4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:20 UTC) from {username='oryx', principal='oryx/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53816
I20260620 07:07:20.972759 16347 raft_consensus.cc:2229] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:20.972795 16348 raft_consensus.cc:2258] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.972821 16346 tablet_replica.cc:330] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: stopping tablet replica
I20260620 07:07:20.972852 16347 raft_consensus.cc:2258] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.972918 16346 raft_consensus.cc:2229] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:20.973217 16346 raft_consensus.cc:2258] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:20.973623 16347 ts_tablet_manager.cc:1905] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.973891 16348 ts_tablet_manager.cc:1905] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.974129 16346 ts_tablet_manager.cc:1905] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:20.976102 16347 ts_tablet_manager.cc:1918] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.976102 16348 ts_tablet_manager.cc:1918] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.976167 16347 log.cc:1196] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/5eac76c4bb6c474084d144013b44fe4a
I20260620 07:07:20.976167 16348 log.cc:1196] T 5eac76c4bb6c474084d144013b44fe4a P 32623289074e4faeb18eef1829883885: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/5eac76c4bb6c474084d144013b44fe4a
I20260620 07:07:20.976162 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 15884
I20260620 07:07:20.979691 16347 ts_tablet_manager.cc:1939] T 5eac76c4bb6c474084d144013b44fe4a P 1a2ed005ebf54d889120b7c05e385d7d: Deleting consensus metadata
I20260620 07:07:20.980551 15796 catalog_manager.cc:4841] TS 1a2ed005ebf54d889120b7c05e385d7d (127.8.24.2:39007): tablet 5eac76c4bb6c474084d144013b44fe4a (table test-table [id=7b9a624cdf9c45a18430a4d8f496ab1d]) successfully deleted
I20260620 07:07:20.981760 16346 ts_tablet_manager.cc:1918] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 07:07:20.981840 16346 log.cc:1196] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/5eac76c4bb6c474084d144013b44fe4a
I20260620 07:07:20.982074 16346 ts_tablet_manager.cc:1939] T 5eac76c4bb6c474084d144013b44fe4a P 948c2ebc37fe408795022c3f1ebab86d: Deleting consensus metadata
I20260620 07:07:20.982826 15799 catalog_manager.cc:4841] TS 948c2ebc37fe408795022c3f1ebab86d (127.8.24.3:40261): tablet 5eac76c4bb6c474084d144013b44fe4a (table test-table [id=7b9a624cdf9c45a18430a4d8f496ab1d]) successfully deleted
W20260620 07:07:20.989199 15797 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:44527 (error 108)
I20260620 07:07:20.989357 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16024
I20260620 07:07:20.990931 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:20.990475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44527 (local address 127.0.0.1:44936)
0620 07:07:20.990759 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:20.990831 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":58,"threads_started":1}
W20260620 07:07:20.991242 15797 catalog_manager.cc:4584] TS 32623289074e4faeb18eef1829883885 (127.8.24.1:44527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5eac76c4bb6c474084d144013b44fe4a: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44527: connect: Connection refused (error 111)
I20260620 07:07:20.995651 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16164
I20260620 07:07:21.001453 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 15780
2026-06-20T07:07:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3285 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:21 dist-test-slave-wjw3 krb5kdc[16425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:21 dist-test-slave-wjw3 krb5kdc[16425](info): set up 2 sockets
Jun 20 07:07:21 dist-test-slave-wjw3 krb5kdc[16425](info): commencing operation
krb5kdc: starting...
W20260620 07:07:21.939831 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:22 dist-test-slave-wjw3 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:22.048190 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40917
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:22.152405 16438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:22.152662 16438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:22.152714 16438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:22.156654 16438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:22.156710 16438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:22.156735 16438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:22.156755 16438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:22.156775 16438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:22.161826 16438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:40917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:22.162832 16438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:22.163836 16438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:22.171615 16444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:22.171645 16443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:22.171615 16446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:22.171689 16438 server_base.cc:1034] running on GCE node
I20260620 07:07:22.172408 16438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:22.173454 16438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:22.174612 16438 hybrid_clock.cc:648] HybridClock initialized: now 1781939242174593 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T07:07:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1184 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:22 dist-test-slave-wjw3 krb5kdc[16451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:22 dist-test-slave-wjw3 krb5kdc[16451](info): set up 2 sockets
Jun 20 07:07:22 dist-test-slave-wjw3 krb5kdc[16451](info): commencing operation
krb5kdc: starting...
W20260620 07:07:23.131822 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:23.237613 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39915
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:23.340361 16464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:23.340602 16464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:23.340641 16464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:23.344497 16464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:23.344545 16464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:23.344563 16464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:23.344578 16464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:23.344594 16464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:23.349598 16464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:23.350514 16464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:23.351406 16464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:23.359061 16469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:23.359202 16464 server_base.cc:1034] running on GCE node
W20260620 07:07:23.359129 16472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:23.359077 16470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:23.359808 16464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:23.360776 16464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:23.361920 16464 hybrid_clock.cc:648] HybridClock initialized: now 1781939243361897 us; error 35 us; skew 500 ppm
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:23.368883 16464 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:23.369709 16464 webserver.cc:460] Webserver started at http://127.8.24.62:37755/ using document root <none> and password file <none>
I20260620 07:07:23.370198 16464 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:23.370252 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:23.370409 16464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:23.372335 16464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "7fd13d0a4df94fd6b0858dbf52143c7d"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "16b8f1103d4162271f75d37e098387e7"
server_key_iv: "5ee42c424aaf987aa6750b406d95dd4d"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.372822 16464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "7fd13d0a4df94fd6b0858dbf52143c7d"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "16b8f1103d4162271f75d37e098387e7"
server_key_iv: "5ee42c424aaf987aa6750b406d95dd4d"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.376227 16464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:23.378376 16479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.379568 16464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:23.379673 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "7fd13d0a4df94fd6b0858dbf52143c7d"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "16b8f1103d4162271f75d37e098387e7"
server_key_iv: "5ee42c424aaf987aa6750b406d95dd4d"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.379760 16464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:23.404812 16464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:23.407585 16464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:23.407763 16464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:23.414225 16464 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39915
I20260620 07:07:23.414250 16531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39915 every 8 connection(s)
I20260620 07:07:23.415166 16464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:23.417809 16532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:23.422356 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16464
I20260620 07:07:23.422456 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:23.422710 8288 external_mini_cluster.cc:1428] Setting key 3c92db3a176b480d355ff95423a9adcd
I20260620 07:07:23.423307 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Bootstrap starting.
I20260620 07:07:23.425858 16532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:23.426698 16532 log.cc:824] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:23.428747 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: No bootstrap required, opened a new log
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:23.432466 16532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:23.432727 16532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:23.432786 16532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd13d0a4df94fd6b0858dbf52143c7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:23.433239 16532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:23.433359 16532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:23.433418 16532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:23.433506 16532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:23.434315 16532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:23.434650 16532 leader_election.cc:302] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fd13d0a4df94fd6b0858dbf52143c7d; no voters:
I20260620 07:07:23.434926 16532 leader_election.cc:288] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:23.435034 16537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:23.435237 16537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 LEADER]: Becoming Leader. State: Replica: 7fd13d0a4df94fd6b0858dbf52143c7d, State: Running, Role: LEADER
I20260620 07:07:23.436121 16532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:23.436125 16537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:23.437129 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:23.424140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.8.24.62:39915)
0620 07:07:23.424558 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:23.424578 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:23.424619 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:07:23.425427 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:23.425449 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:23.425852 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:23.426225 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:23.426840 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.428210 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.429111 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.429764 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.429913 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:23.432126 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:23.432132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:23.432147 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:23.432176 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:23.433934 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.434318 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.434324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.434329 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.434491 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.434736 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.434739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.434741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.435268 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:23.435491 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:23.435789 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 07:07:23.436100 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260620 07:07:23.437630 16539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fd13d0a4df94fd6b0858dbf52143c7d. Latest consensus state: current_term: 1 leader_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } } }
I20260620 07:07:23.437741 16539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:23.438313 16538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } } }
I20260620 07:07:23.438747 16538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: This master's current role is: LEADER
W20260620 07:07:23.441537 16552 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:07:23.441608 16552 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:07:23.441684 16551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:23.442509 16551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:23.447751 16551 catalog_manager.cc:1347] Generated new cluster ID: 12bd033a9d3c4818af812b6314a7078c
I20260620 07:07:23.447808 16551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:23.459172 16551 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:23.460405 16551 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:23.467061 16551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Generated new TSK 0
I20260620 07:07:23.467823 16551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:23.494190 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:23.600946 16558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:23.601207 16558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:23.601263 16558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:23.605226 16558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:23.605293 16558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:23.605370 16558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:23.610528 16558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:23.611552 16558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:23.612525 16558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:23.621366 16563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:23.621388 16564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:23.621357 16566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:23.621723 16558 server_base.cc:1034] running on GCE node
I20260620 07:07:23.622107 16558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:23.622697 16558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:23.623842 16558 hybrid_clock.cc:648] HybridClock initialized: now 1781939243623827 us; error 28 us; skew 500 ppm
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:23.626547 16558 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:23.627403 16558 webserver.cc:460] Webserver started at http://127.8.24.1:35943/ using document root <none> and password file <none>
I20260620 07:07:23.628001 16558 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:23.628065 16558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:23.628253 16558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:23.630268 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "a727e1662049418fb68155ca64505c19"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "6f0c132d84f76768d7808bf065efc7f8"
server_key_iv: "f88d243664b23fe1b959504875ea87fa"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.630790 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "a727e1662049418fb68155ca64505c19"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "6f0c132d84f76768d7808bf065efc7f8"
server_key_iv: "f88d243664b23fe1b959504875ea87fa"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.634343 16558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260620 07:07:23.636726 16573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.637974 16558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:23.638073 16558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "a727e1662049418fb68155ca64505c19"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "6f0c132d84f76768d7808bf065efc7f8"
server_key_iv: "f88d243664b23fe1b959504875ea87fa"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.638163 16558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:23.662972 16558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:23.665851 16558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:23.666023 16558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:23.666597 16558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:23.667383 16558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:23.667460 16558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.667523 16558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:23.667554 16558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.676092 16558 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43971
I20260620 07:07:23.676118 16686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43971 every 8 connection(s)
I20260620 07:07:23.676923 16558 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:23.679986 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16558
I20260620 07:07:23.680099 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:23.680349 8288 external_mini_cluster.cc:1428] Setting key 45263907aedd4d42fdaaa1da4fc5edd2
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:07:23.690707 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:23.678688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:35713 (local address 127.8.24.62:39915)
0620 07:07:23.678814 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:23.678818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:23.679843 (+ 1025us) server_negotiation.cc:407] Connection header received
0620 07:07:23.680777 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:23.680780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:23.680832 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:23.680906 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:23.682228 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.682772 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.684372 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.684601 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.684683 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:23.687204 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:23.687207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:23.687209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:23.687231 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:23.688725 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.689277 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.689279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.689281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.689354 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.689663 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.689666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.689668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.689842 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:23.689925 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:23.690413 (+ 488us) server_negotiation.cc:299] Negotiation successful
0620 07:07:23.690535 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:23.691771 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:23.679061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:35713)
0620 07:07:23.679706 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:23.679745 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:23.680557 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:23.681098 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:23.681107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:23.681552 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:23.682098 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:23.682112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.682900 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.682903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:23.684253 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:23.684264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.684734 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.684737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:23.684914 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:23.685501 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:23.685520 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:23.687045 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:23.688847 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:23.688853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:23.688869 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:23.689179 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:23.689435 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:23.689438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:23.689440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:23.689575 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:23.689940 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:23.689947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:23.690190 (+ 243us) client_negotiation.cc:769] Sending connection context
0620 07:07:23.690388 (+ 198us) client_negotiation.cc:240] Negotiation successful
0620 07:07:23.690774 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":109,"threads_started":1}
I20260620 07:07:23.692876 16687 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39915
I20260620 07:07:23.693091 16687 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:23.693531 16687 heartbeater.cc:511] Master 127.8.24.62:39915 requested a full tablet report, sending...
I20260620 07:07:23.695055 16496 ts_manager.cc:194] Registered new tserver with Master: a727e1662049418fb68155ca64505c19 (127.8.24.1:43971)
I20260620 07:07:23.696611 16496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:35713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:23.704496 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:23.809036 16692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:23.809283 16692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:23.809325 16692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:23.813179 16692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:23.813234 16692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:23.813292 16692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:23.818348 16692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:23.819281 16692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:23.820250 16692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:23.829087 16698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:23.829166 16697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:23.829378 16692 server_base.cc:1034] running on GCE node
W20260620 07:07:23.829169 16700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:23.829744 16692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:23.830323 16692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:23.831496 16692 hybrid_clock.cc:648] HybridClock initialized: now 1781939243831466 us; error 39 us; skew 500 ppm
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:23.833956 16692 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:23.834753 16692 webserver.cc:460] Webserver started at http://127.8.24.2:39349/ using document root <none> and password file <none>
I20260620 07:07:23.835325 16692 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:23.835400 16692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:23.835696 16692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:23.837899 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "318de2083b28497586222163f17c3a12"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "ae4f6932a9da0cb58a700337dd80af36"
server_key_iv: "8af56b2beaeae0798b1400ea2dfa7ca6"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.838433 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "318de2083b28497586222163f17c3a12"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "ae4f6932a9da0cb58a700337dd80af36"
server_key_iv: "8af56b2beaeae0798b1400ea2dfa7ca6"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.841964 16692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:23.844203 16707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.845286 16692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:23.845384 16692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "318de2083b28497586222163f17c3a12"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "ae4f6932a9da0cb58a700337dd80af36"
server_key_iv: "8af56b2beaeae0798b1400ea2dfa7ca6"
server_key_version: "encryptionkey@0"
I20260620 07:07:23.845468 16692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:23.858980 16692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:23.861477 16692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:23.861642 16692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:23.862188 16692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:23.862953 16692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:23.862998 16692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.863047 16692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:23.863075 16692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:23.871084 16692 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:42381
I20260620 07:07:23.871110 16820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:42381 every 8 connection(s)
I20260620 07:07:23.871899 16692 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:23.879760 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16692
I20260620 07:07:23.879869 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:23.880117 8288 external_mini_cluster.cc:1428] Setting key 8465431883f0269fa05a291df7aa851c
Jun 20 07:07:23 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939243, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:23.884554 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:23.873536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:34047 (local address 127.8.24.62:39915)
0620 07:07:23.873689 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:23.873693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:23.874664 (+ 971us) server_negotiation.cc:407] Connection header received
0620 07:07:23.875493 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:23.875496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:23.875538 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:23.875604 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:23.876636 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.877147 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.878265 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.878482 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.878554 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:23.880869 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:23.880873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:23.880876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:23.880901 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:23.882341 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.882787 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.882790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.882792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.882873 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:23.883224 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:23.883226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:23.883227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:23.883448 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:23.883539 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:23.884251 (+ 712us) server_negotiation.cc:299] Negotiation successful
0620 07:07:23.884417 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 07:07:23.885430 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:23.873913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:34047)
0620 07:07:23.874527 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:23.874567 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:23.875305 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:23.875735 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:23.875742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:23.876078 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:23.876529 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:23.876542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.877269 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.877273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:23.878162 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:23.878171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:23.878589 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:23.878592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:23.878732 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:23.879235 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:23.879251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:23.880717 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:23.882436 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:23.882442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:23.882455 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:23.882705 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:23.882967 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:23.882970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:23.882972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:23.883145 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:23.883559 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:23.883565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:23.883885 (+ 320us) client_negotiation.cc:769] Sending connection context
0620 07:07:23.884145 (+ 260us) client_negotiation.cc:240] Negotiation successful
0620 07:07:23.884455 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":108,"threads_started":1}
I20260620 07:07:23.886528 16821 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39915
I20260620 07:07:23.886741 16821 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:23.887185 16821 heartbeater.cc:511] Master 127.8.24.62:39915 requested a full tablet report, sending...
I20260620 07:07:23.888177 16496 ts_manager.cc:194] Registered new tserver with Master: 318de2083b28497586222163f17c3a12 (127.8.24.2:42381)
I20260620 07:07:23.888715 16496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:34047
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:23.903644 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:24.007794 16826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:24.008199 16826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:24.008280 16826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:24.012423 16826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:24.012480 16826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:24.012559 16826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:24.017663 16826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:24.018711 16826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:24.019713 16826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:24.028514 16831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.028584 16832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.028626 16826 server_base.cc:1034] running on GCE node
W20260620 07:07:24.028685 16834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.029163 16826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:24.029736 16826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:24.030884 16826 hybrid_clock.cc:648] HybridClock initialized: now 1781939244030864 us; error 34 us; skew 500 ppm
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:24.033360 16826 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:24.034129 16826 webserver.cc:460] Webserver started at http://127.8.24.3:42519/ using document root <none> and password file <none>
I20260620 07:07:24.034641 16826 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:24.034701 16826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:24.034873 16826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:24.036939 16826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "23419410144e41f590d0ed93a9b9bd7d"
format_stamp: "Formatted at 2026-06-20 07:07:24 on dist-test-slave-wjw3"
server_key: "a16e5bd2c80d515aef74ef8bc86d1165"
server_key_iv: "d731ff1c4ce9fdfe2ffc12cb26bf28f6"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.037453 16826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "23419410144e41f590d0ed93a9b9bd7d"
format_stamp: "Formatted at 2026-06-20 07:07:24 on dist-test-slave-wjw3"
server_key: "a16e5bd2c80d515aef74ef8bc86d1165"
server_key_iv: "d731ff1c4ce9fdfe2ffc12cb26bf28f6"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.040910 16826 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260620 07:07:24.043207 16841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.044461 16826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:24.044559 16826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "23419410144e41f590d0ed93a9b9bd7d"
format_stamp: "Formatted at 2026-06-20 07:07:24 on dist-test-slave-wjw3"
server_key: "a16e5bd2c80d515aef74ef8bc86d1165"
server_key_iv: "d731ff1c4ce9fdfe2ffc12cb26bf28f6"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.044651 16826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:24.055269 16826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:24.057898 16826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:24.058060 16826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:24.058604 16826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:24.059366 16826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:24.059415 16826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.059527 16826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:24.059559 16826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.067468 16826 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40701
I20260620 07:07:24.067500 16954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40701 every 8 connection(s)
I20260620 07:07:24.068317 16826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:24.068578 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16826
I20260620 07:07:24.068677 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:24.068953 8288 external_mini_cluster.cc:1428] Setting key 8b4471f8e2277b70c55ec5a1e2473b4f
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:24.080178 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.070153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44253 (local address 127.8.24.62:39915)
0620 07:07:24.070263 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.070267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.071409 (+ 1142us) server_negotiation.cc:407] Connection header received
0620 07:07:24.072244 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.072246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.072291 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.072351 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.073395 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.073911 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.074973 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.075178 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.075248 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.077361 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:24.077364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:24.077366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:24.077388 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:24.078668 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:24.079029 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:24.079032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:24.079033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:24.079094 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:24.079317 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:24.079319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:24.079320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:24.079521 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:24.079592 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:24.079973 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 07:07:24.080060 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 07:07:24.080878 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.070628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:44253)
0620 07:07:24.071284 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.071321 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.072085 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.072495 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.072501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.072836 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.073297 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.073309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.073996 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.073998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.074876 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.074884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.075279 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.075282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.075409 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.075931 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.075945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.077223 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:24.078746 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:24.078750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:24.078760 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:24.078966 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:24.079150 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:24.079152 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:24.079153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:24.079261 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:24.079595 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:24.079600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:24.079801 (+ 201us) client_negotiation.cc:769] Sending connection context
0620 07:07:24.079940 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 07:07:24.080158 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":128,"threads_started":1}
I20260620 07:07:24.081859 16955 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39915
I20260620 07:07:24.082062 16955 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:24.082578 16955 heartbeater.cc:511] Master 127.8.24.62:39915 requested a full tablet report, sending...
I20260620 07:07:24.083590 16495 ts_manager.cc:194] Registered new tserver with Master: 23419410144e41f590d0ed93a9b9bd7d (127.8.24.3:40701)
I20260620 07:07:24.084138 16495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:44253
I20260620 07:07:24.093729 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
Entry for principal oryx/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab.
WARNING: no policy specified for oryx/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
Entry for principal oryx/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab.
WARNING: no policy specified for oryx/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
Entry for principal oryx/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab.
I20260620 07:07:24.151033 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16558
I20260620 07:07:24.155875 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16692
I20260620 07:07:24.160128 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16826
I20260620 07:07:24.164386 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16464
I20260620 07:07:24.169493 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39915
--webserver_interface=127.8.24.62
--webserver_port=37755
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:24.277089 16964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:24.277347 16964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:24.277384 16964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:24.281286 16964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:24.281338 16964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:24.281355 16964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:24.281378 16964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:24.281391 16964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:24.286574 16964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:24.287633 16964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:24.288602 16964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:24.296798 16970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.296902 16964 server_base.cc:1034] running on GCE node
W20260620 07:07:24.296809 16972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.296792 16969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.297595 16964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:24.298681 16964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:24.299829 16964 hybrid_clock.cc:648] HybridClock initialized: now 1781939244299808 us; error 34 us; skew 500 ppm
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:24.302407 16964 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:24.303254 16964 webserver.cc:460] Webserver started at http://127.8.24.62:37755/ using document root <none> and password file <none>
I20260620 07:07:24.303838 16964 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:24.303938 16964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:24.307690 16964 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 07:07:24.309633 16979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.310767 16964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:24.310866 16964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "7fd13d0a4df94fd6b0858dbf52143c7d"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "16b8f1103d4162271f75d37e098387e7"
server_key_iv: "5ee42c424aaf987aa6750b406d95dd4d"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.311381 16964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:24.324991 16964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:24.327832 16964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:24.328003 16964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:24.334446 16964 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39915
I20260620 07:07:24.334486 17031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39915 every 8 connection(s)
I20260620 07:07:24.335479 16964 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:24.339613 17032 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:07:24.341207 17032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Bootstrap starting.
I20260620 07:07:24.344599 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 16964
I20260620 07:07:24.345010 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:43971
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:39915
--webserver_port=35943
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:07:24.357143 17032 log.cc:824] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:24.363466 17032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:07:24.363775 17032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Bootstrap complete.
I20260620 07:07:24.368052 17032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:24.368425 17032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd13d0a4df94fd6b0858dbf52143c7d, State: Initialized, Role: FOLLOWER
I20260620 07:07:24.368886 17032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:24.369009 17032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:24.369071 17032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:24.369163 17032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:07:24.370954 17032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:24.371249 17032 leader_election.cc:302] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fd13d0a4df94fd6b0858dbf52143c7d; no voters:
I20260620 07:07:24.371578 17032 leader_election.cc:288] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 07:07:24.371668 17036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260620 07:07:24.371846 17036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [term 2 LEADER]: Becoming Leader. State: Replica: 7fd13d0a4df94fd6b0858dbf52143c7d, State: Running, Role: LEADER
I20260620 07:07:24.372123 17036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } }
I20260620 07:07:24.372727 17032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:24.372982 17036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fd13d0a4df94fd6b0858dbf52143c7d. Latest consensus state: current_term: 2 leader_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } } }
I20260620 07:07:24.373085 17036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:24.373417 17040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:24.373512 17037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd13d0a4df94fd6b0858dbf52143c7d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39915 } } }
I20260620 07:07:24.373675 17037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:24.376566 17040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:24.377166 17040 catalog_manager.cc:1259] Loaded cluster ID: 12bd033a9d3c4818af812b6314a7078c
I20260620 07:07:24.377223 17040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:24.379581 17040 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:24.380229 17040 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7fd13d0a4df94fd6b0858dbf52143c7d: Loaded TSK: 0
I20260620 07:07:24.381376 17040 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:07:24.452306 17034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:24.452549 17034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:24.452615 17034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:24.456544 17034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:24.456611 17034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:24.456687 17034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:24.461802 17034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.1.keytab
--principal=oryx/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:43971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=35943
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:24.462720 17034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:24.463673 17034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:24.473520 17059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.473521 17058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.473776 17061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.473965 17034 server_base.cc:1034] running on GCE node
I20260620 07:07:24.474351 17034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:24.474987 17034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:24.476133 17034 hybrid_clock.cc:648] HybridClock initialized: now 1781939244476100 us; error 51 us; skew 500 ppm
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:24.478797 17034 init.cc:377] Logged in from keytab as oryx/127.8.24.1@KRBTEST.COM (short username oryx)
I20260620 07:07:24.479692 17034 webserver.cc:460] Webserver started at http://127.8.24.1:35943/ using document root <none> and password file <none>
I20260620 07:07:24.480260 17034 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:24.480336 17034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:24.484117 17034 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:24.486145 17068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.487398 17034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:24.487519 17034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "a727e1662049418fb68155ca64505c19"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "6f0c132d84f76768d7808bf065efc7f8"
server_key_iv: "f88d243664b23fe1b959504875ea87fa"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.488018 17034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:24.508499 17034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:24.511353 17034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:24.511543 17034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:24.512101 17034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:24.512878 17034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:24.512923 17034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.512972 17034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:24.513000 17034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.521216 17034 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43971
I20260620 07:07:24.521270 17181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43971 every 8 connection(s)
I20260620 07:07:24.522185 17034 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:24.530859 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17034
I20260620 07:07:24.531256 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:42381
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:39915
--webserver_port=39349
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 07:07:24.534512 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.524177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57325 (local address 127.8.24.62:39915)
0620 07:07:24.524652 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.524672 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.525067 (+ 395us) server_negotiation.cc:407] Connection header received
0620 07:07:24.526113 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.526125 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.526402 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.526686 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.527824 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.528808 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.529976 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.530446 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.530590 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.533781 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:57325: BlockingRecv error: recv got EOF from 127.8.24.1:57325 (error 108)
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
I20260620 07:07:24.534520 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.524276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:57325)
0620 07:07:24.524917 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.524958 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.525729 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.526784 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.526792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.527185 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.527669 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.527683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.528936 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.528939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.529855 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.529865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.530598 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.530603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.530787 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.531326 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.531347 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.533555 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":126,"threads_started":1}
W20260620 07:07:24.534992 17182 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39915 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:39915: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.540354 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.535386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:45571)
0620 07:07:24.535562 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.535573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.535629 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.535838 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.535840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.535967 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.536055 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.536059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.537007 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.537009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.537535 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.537541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.538056 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.538058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.538117 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.538561 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.538571 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.540262 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112}
W20260620 07:07:24.540534 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.535480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:45571 (local address 127.8.24.62:39915)
0620 07:07:24.535623 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.535627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.535644 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:07:24.535688 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.535690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.535740 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.535834 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.536150 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.536910 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.537633 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.537958 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.538058 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.540424 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:45571: BlockingRecv error: recv got EOF from 127.8.24.1:45571 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.545872 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.540783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:34513)
0620 07:07:24.541042 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.541052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.541116 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.541292 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.541294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.541399 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.541462 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.541466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.542545 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.542548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.543161 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.543169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.543708 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.543710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.543764 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.544189 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.544199 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.545776 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":82,"threads_started":1}
W20260620 07:07:24.546013 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.540864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34513 (local address 127.8.24.62:39915)
0620 07:07:24.540974 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.540977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.541100 (+ 123us) server_negotiation.cc:407] Connection header received
0620 07:07:24.541170 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.541173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.541213 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.541284 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.541540 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.542385 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.543265 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.543622 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.543713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.545910 (+ 2197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:34513: BlockingRecv error: recv got EOF from 127.8.24.1:34513 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 07:07:24.546118 17182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:07:24.637238 17186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:24.637503 17186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:24.637547 17186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:24.641503 17186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:24.641556 17186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:24.641623 17186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:24.646834 17186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.2.keytab
--principal=oryx/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:42381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=39349
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:24.647845 17186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:24.648795 17186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:24.657492 17193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.657467 17195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.657513 17192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.657684 17186 server_base.cc:1034] running on GCE node
I20260620 07:07:24.658129 17186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:24.658696 17186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:24.659885 17186 hybrid_clock.cc:648] HybridClock initialized: now 1781939244659859 us; error 34 us; skew 500 ppm
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:24.662309 17186 init.cc:377] Logged in from keytab as oryx/127.8.24.2@KRBTEST.COM (short username oryx)
I20260620 07:07:24.663115 17186 webserver.cc:460] Webserver started at http://127.8.24.2:39349/ using document root <none> and password file <none>
I20260620 07:07:24.663673 17186 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:24.663753 17186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:24.667426 17186 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:24.669405 17202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.670547 17186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:24.670645 17186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "318de2083b28497586222163f17c3a12"
format_stamp: "Formatted at 2026-06-20 07:07:23 on dist-test-slave-wjw3"
server_key: "ae4f6932a9da0cb58a700337dd80af36"
server_key_iv: "8af56b2beaeae0798b1400ea2dfa7ca6"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.671155 17186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:24.687295 17186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:24.689913 17186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:24.690093 17186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:24.690670 17186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:24.691500 17186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:24.691551 17186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.691623 17186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:24.691653 17186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.699747 17186 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:42381
I20260620 07:07:24.699785 17315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:42381 every 8 connection(s)
I20260620 07:07:24.700693 17186 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:24.706663 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17186
I20260620 07:07:24.707047 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:40701
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:39915
--webserver_port=42519
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 07:07:24.710932 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.702349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:41105 (local address 127.8.24.62:39915)
0620 07:07:24.702479 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.702483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.703475 (+ 992us) server_negotiation.cc:407] Connection header received
0620 07:07:24.704378 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.704382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.704440 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.704510 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.705629 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.706143 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.707301 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.707565 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.707651 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.710811 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:41105: BlockingRecv error: recv got EOF from 127.8.24.2:41105 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:07:24.711463 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.702722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:41105)
0620 07:07:24.703333 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.703372 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.704188 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.704694 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.704701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.705028 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.705493 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.705506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.706251 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.706255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.707164 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.707174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.707704 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.707707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.707861 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.708405 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.708425 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.710663 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":122,"threads_started":1}
W20260620 07:07:24.711850 17316 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39915 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:39915: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.716598 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.712263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:43519)
0620 07:07:24.712368 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.712381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.712447 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.712657 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.712660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.712788 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.712863 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.712868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.713509 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.713513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.714022 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.714029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.714461 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.714464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.714555 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.714936 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.714947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.716488 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 07:07:24.716686 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.712335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:43519 (local address 127.8.24.62:39915)
0620 07:07:24.712442 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.712445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.712458 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:24.712528 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.712530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.712576 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.712635 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.712961 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.713423 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.714126 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.714355 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.714470 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.716593 (+ 2123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:43519: BlockingRecv error: recv got EOF from 127.8.24.2:43519 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.721606 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.717139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:38895)
0620 07:07:24.717237 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.717249 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.717308 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.717507 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.717509 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.717617 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.717677 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.717681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.718596 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.718598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.719089 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.719094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.719612 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.719614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.719665 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.720018 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.720028 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.721532 (+ 1504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 07:07:24.721751 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.717205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:38895 (local address 127.8.24.62:39915)
0620 07:07:24.717311 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.717315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.717350 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:07:24.717386 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.717388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.717426 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.717487 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.717758 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.718516 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.719170 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.719530 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.719619 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.721652 (+ 2033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:38895: BlockingRecv error: recv got EOF from 127.8.24.2:38895 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 07:07:24.722002 17316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:07:24.815153 17319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:24.815486 17319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:24.815557 17319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:24.819532 17319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:24.819586 17319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:24.819640 17319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:24.824813 17319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/krb5kdc/oryx_127.8.24.3.keytab
--principal=oryx/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:40701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=42519
--tserver_master_addrs=127.8.24.62:39915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:24.825793 17319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:24.826728 17319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:24.835667 17325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:24.835630 17324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.835763 17319 server_base.cc:1034] running on GCE node
W20260620 07:07:24.835633 17327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:24.836315 17319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:24.836922 17319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:24.838078 17319 hybrid_clock.cc:648] HybridClock initialized: now 1781939244838060 us; error 32 us; skew 500 ppm
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939244, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:24.840557 17319 init.cc:377] Logged in from keytab as oryx/127.8.24.3@KRBTEST.COM (short username oryx)
I20260620 07:07:24.841348 17319 webserver.cc:460] Webserver started at http://127.8.24.3:42519/ using document root <none> and password file <none>
I20260620 07:07:24.841847 17319 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:24.841921 17319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:24.845561 17319 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:24.847424 17334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.848579 17319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:24.848672 17319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "23419410144e41f590d0ed93a9b9bd7d"
format_stamp: "Formatted at 2026-06-20 07:07:24 on dist-test-slave-wjw3"
server_key: "a16e5bd2c80d515aef74ef8bc86d1165"
server_key_iv: "d731ff1c4ce9fdfe2ffc12cb26bf28f6"
server_key_version: "encryptionkey@0"
I20260620 07:07:24.849169 17319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:24.865854 17319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:24.868398 17319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:24.868546 17319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:24.869059 17319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:24.869773 17319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:24.869815 17319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.869855 17319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:24.869884 17319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:24.878291 17319 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40701
I20260620 07:07:24.878316 17447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40701 every 8 connection(s)
I20260620 07:07:24.879201 17319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:24.883049 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17319
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 07:07:24.889315 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.880826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:44043 (local address 127.8.24.62:39915)
0620 07:07:24.880952 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.880957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.882002 (+ 1045us) server_negotiation.cc:407] Connection header received
0620 07:07:24.882844 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.882848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.882901 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.882975 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.884258 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.884772 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.885833 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.886041 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.886123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.889192 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:44043: BlockingRecv error: recv got EOF from 127.8.24.3:44043 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:07:24.889796 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.881255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:44043)
0620 07:07:24.881846 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.881885 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.882657 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.883164 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.883174 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.883572 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.884126 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.884139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.884881 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.884884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.885725 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.885733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.886164 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.886167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.886317 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.886867 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.886887 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.889054 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":103,"threads_started":1}
W20260620 07:07:24.890206 17448 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39915 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.24.62:39915: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
I20260620 07:07:24.891800 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.883939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.8.24.62:39915)
0620 07:07:24.884126 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.884129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.884140 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:24.884172 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.884174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.884206 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.884279 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.884948 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.885453 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.886269 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.886487 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.886579 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.887785 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:24.887792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:24.887808 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:24.887838 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:24.889477 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:24.889853 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:24.889858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:24.889865 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:24.890005 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:24.890597 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:24.890600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:24.890602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:24.891009 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:24.891189 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:24.891466 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 07:07:24.891688 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.895231 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.890559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:47583)
0620 07:07:24.890641 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.890654 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.890720 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.890896 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.890899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.891038 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.891118 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.891123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.891859 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.891862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.892452 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.892461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.892847 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.892850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.892931 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.893290 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.893303 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.895120 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260620 07:07:24.895323 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.890636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47583 (local address 127.8.24.62:39915)
0620 07:07:24.890731 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.890734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.890746 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:24.890777 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.890780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.890815 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.890874 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.891216 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.891771 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.892553 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.892761 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.892841 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.895235 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:47583: BlockingRecv error: recv got EOF from 127.8.24.3:47583 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:24 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:24.899503 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:24.895772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:49015)
0620 07:07:24.895877 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:24.895890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:24.895956 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:24.896119 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:24.896121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:24.896226 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:24.896287 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.896291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.896866 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.896868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.897359 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:24.897365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.897677 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.897680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:24.897730 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.898078 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:24.898089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:24.899391 (+ 1302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 07:07:24.899629 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:24.895838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49015 (local address 127.8.24.62:39915)
0620 07:07:24.895956 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:24.895959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:24.895970 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:24.896014 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:24.896016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:24.896046 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:24.896102 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:24.896356 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.896795 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.897431 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:24.897613 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:24.897673 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:24.899528 (+ 1855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:49015: BlockingRecv error: recv got EOF from 127.8.24.3:49015 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 07:07:24.899837 17448 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:25.551710 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:25.546736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:46985)
0620 07:07:25.547016 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:25.547031 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:25.547141 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:25.547383 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:25.547385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:25.547580 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:25.547674 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.547680 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.548370 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.548372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.548936 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.548943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.549320 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.549322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.549376 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.549864 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:25.549875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:25.551516 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
W20260620 07:07:25.551817 17454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:25.546819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46985 (local address 127.8.24.62:39915)
0620 07:07:25.547107 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:25.547111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:25.547128 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:07:25.547235 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:25.547238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:25.547285 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:25.547358 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:25.547763 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.548290 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.549024 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.549240 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.549311 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.551661 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:46985: BlockingRecv error: recv got EOF from 127.8.24.1:46985 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":108,"threads_started":1}
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:25.727496 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:25.722490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:42511)
0620 07:07:25.722749 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:25.722763 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:25.722867 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:25.723093 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:25.723096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:25.723282 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:25.723371 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.723377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.724137 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.724141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.724720 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.724728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.725146 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.725149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.725231 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.725672 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:25.725684 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:25.727337 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
W20260620 07:07:25.727579 17454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:25.722591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:42511 (local address 127.8.24.62:39915)
0620 07:07:25.722733 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:25.722738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:25.722827 (+ 89us) server_negotiation.cc:407] Connection header received
0620 07:07:25.722948 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:25.722951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:25.722997 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:25.723063 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:25.723522 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.724043 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.724829 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.725054 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.725136 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.727486 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:42511: BlockingRecv error: recv got EOF from 127.8.24.2:42511 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:25 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:25.907855 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:25.900387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:46509)
0620 07:07:25.900648 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:25.900662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:25.900757 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:25.900957 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:25.900959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:25.901122 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:25.901213 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.901218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.901891 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.901893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.902467 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:25.902473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.902894 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.902898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:25.902972 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.906051 (+ 3079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:25.906073 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:25.907701 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
W20260620 07:07:25.907934 17454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:25.900460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46509 (local address 127.8.24.62:39915)
0620 07:07:25.900593 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:25.900597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:25.900725 (+ 128us) server_negotiation.cc:407] Connection header received
0620 07:07:25.900831 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:25.900833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:25.900878 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:25.900938 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:25.901300 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.901813 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.902581 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:25.902799 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:25.902883 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:25.907833 (+ 4950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:46509: BlockingRecv error: recv got EOF from 127.8.24.3:46509 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:26.557685 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:26.552628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:40449)
0620 07:07:26.552960 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:26.552978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:26.553081 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:26.553331 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:26.553334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:26.553515 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:26.553615 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.553620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.554338 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.554341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.554928 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.554937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.555339 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.555341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.555430 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.555911 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:26.555923 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:26.557546 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
W20260620 07:07:26.557835 17458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:26.552744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:40449 (local address 127.8.24.62:39915)
0620 07:07:26.553017 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:26.553020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:26.553033 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:26.553158 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:26.553160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:26.553203 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:26.553280 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:26.553709 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.554237 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.555021 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.555243 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.555314 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.557700 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:40449: BlockingRecv error: recv got EOF from 127.8.24.1:40449 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":83,"threads_started":1}
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:26.733481 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:26.728264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:50855)
0620 07:07:26.728502 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:26.728514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:26.728615 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:26.728816 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:26.728818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:26.728982 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:26.729077 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.729082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.729782 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.729786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.730495 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.730508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.730984 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.730988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.731071 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.731581 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:26.731594 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:26.733312 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
W20260620 07:07:26.733564 17458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:26.728339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:50855 (local address 127.8.24.62:39915)
0620 07:07:26.728460 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:26.728463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:26.728572 (+ 109us) server_negotiation.cc:407] Connection header received
0620 07:07:26.728684 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:26.728686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:26.728729 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:26.728793 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:26.729157 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.729662 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.730611 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.730874 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.730976 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.733463 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:50855: BlockingRecv error: recv got EOF from 127.8.24.2:50855 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:26 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:26.913518 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:26.908539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:38199)
0620 07:07:26.908806 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:26.908819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:26.908907 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:26.909106 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:26.909107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:26.909269 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:26.909388 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.909393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.910215 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.910219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.910827 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:26.910837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.911246 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.911248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:26.911308 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.911758 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:26.911770 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:26.913391 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
W20260620 07:07:26.913609 17458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:26.908724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38199 (local address 127.8.24.62:39915)
0620 07:07:26.908845 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:26.908848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:26.908861 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:26.908977 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:26.908980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:26.909021 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:26.909081 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:26.909498 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.910102 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.910940 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:26.911160 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:26.911232 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:26.913514 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:38199: BlockingRecv error: recv got EOF from 127.8.24.3:38199 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:27.563117 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:27.558368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:54487)
0620 07:07:27.558628 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:27.558641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:27.558723 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:27.558933 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:27.558935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:27.559097 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:27.559185 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.559190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.559891 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.559894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.560441 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.560449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.560841 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.560844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.560899 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.561345 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:27.561356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:27.562993 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
W20260620 07:07:27.563238 17462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:27.558459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:54487 (local address 127.8.24.62:39915)
0620 07:07:27.558695 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:27.558699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:27.558711 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:27.558795 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:27.558797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:27.558839 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:27.558912 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:27.559264 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.559805 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.560538 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.560761 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.560835 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.563126 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:54487: BlockingRecv error: recv got EOF from 127.8.24.1:54487 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:27.739336 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:27.734103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:51301)
0620 07:07:27.734344 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:27.734373 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:27.734479 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:27.734742 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:27.734745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:27.734975 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:27.735096 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.735103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.736045 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.736048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.736650 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.736659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.737031 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.737033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.737094 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.737516 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:27.737526 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:27.739167 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
W20260620 07:07:27.739398 17462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:27.734195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51301 (local address 127.8.24.62:39915)
0620 07:07:27.734376 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:27.734381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:27.734425 (+ 44us) server_negotiation.cc:407] Connection header received
0620 07:07:27.734571 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:27.734574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:27.734621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:27.734692 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:27.735219 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.735924 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.736747 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.736953 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.737019 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.739301 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:51301: BlockingRecv error: recv got EOF from 127.8.24.2:51301 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:27 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:27.919040 17464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:27.914200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:60253)
0620 07:07:27.914461 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:27.914473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:27.914564 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:27.914756 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:27.914758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:27.914943 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:27.915036 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.915041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.915764 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.915767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.916312 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:27.916319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.916690 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.916692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:27.916747 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.917196 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:27.917206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:27.918869 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":68,"threads_started":1}
W20260620 07:07:27.919098 17462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:27.914248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60253 (local address 127.8.24.62:39915)
0620 07:07:27.914385 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:27.914390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:27.914521 (+ 131us) server_negotiation.cc:407] Connection header received
0620 07:07:27.914631 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:27.914633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:27.914675 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:27.914734 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:27.915117 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.915685 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.916397 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:27.916614 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:27.916683 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:27.919000 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:60253: BlockingRecv error: recv got EOF from 127.8.24.3:60253 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:28.573305 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:28.563875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:56413)
0620 07:07:28.564152 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:28.564166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:28.564259 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:28.564487 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:28.564489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:28.564670 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:28.564766 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.564771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.565447 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.565450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.566014 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.566022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.566408 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.566412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.566502 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.571467 (+ 4965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:28.571487 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:28.573173 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":83,"threads_started":1}
W20260620 07:07:28.573457 17466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:28.563965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:56413 (local address 127.8.24.62:39915)
0620 07:07:28.564184 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:28.564187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:28.564215 (+ 28us) server_negotiation.cc:407] Connection header received
0620 07:07:28.564340 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:28.564342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:28.564386 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:28.564458 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:28.564848 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.565355 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.566096 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.566297 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.566360 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.573305 (+ 6945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:56413: BlockingRecv error: recv got EOF from 127.8.24.1:56413 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":57,"threads_started":1}
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:28.745041 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:28.740156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:36379)
0620 07:07:28.740409 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:28.740423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:28.740530 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:28.740773 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:28.740777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:28.740965 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:28.741054 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.741061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.741778 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.741781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.742347 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.742355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.742772 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.742776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.742853 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.743301 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:28.743312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:28.744899 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
W20260620 07:07:28.745148 17466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:28.740217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:36379 (local address 127.8.24.62:39915)
0620 07:07:28.740344 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:28.740349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:28.740488 (+ 139us) server_negotiation.cc:407] Connection header received
0620 07:07:28.740627 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:28.740630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:28.740679 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:28.740742 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:28.741165 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.741683 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.742461 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.742680 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.742761 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.745039 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:36379: BlockingRecv error: recv got EOF from 127.8.24.2:36379 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:28 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:28.924692 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:28.919836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:59105)
0620 07:07:28.920115 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:28.920127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:28.920233 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:28.920474 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:28.920477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:28.920673 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:28.920764 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.920770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.921533 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.921536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.922094 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:28.922102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.922503 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.922507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:28.922597 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.923028 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:28.923039 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:28.924545 (+ 1506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
W20260620 07:07:28.924773 17466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:28.919900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:59105 (local address 127.8.24.62:39915)
0620 07:07:28.920026 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:28.920030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:28.920193 (+ 163us) server_negotiation.cc:407] Connection header received
0620 07:07:28.920325 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:28.920328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:28.920371 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:28.920434 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:28.920882 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.921433 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.922201 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:28.922412 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:28.922493 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:28.924674 (+ 2181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:59105: BlockingRecv error: recv got EOF from 127.8.24.3:59105 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:29.579527 17469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:29.574114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:37455)
0620 07:07:29.574407 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:29.574421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:29.574528 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:29.574914 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:29.574916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:29.575152 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:29.575251 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.575257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.576199 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.576202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.576761 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.576768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.577219 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.577221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.577278 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.577725 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:29.577736 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:29.579360 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
W20260620 07:07:29.579640 17470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:29.574215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:37455 (local address 127.8.24.62:39915)
0620 07:07:29.574491 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:29.574496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:29.574514 (+ 18us) server_negotiation.cc:407] Connection header received
0620 07:07:29.574626 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:29.574629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:29.574682 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:29.574773 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:29.575357 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.576105 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.576891 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.577131 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.577218 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.579526 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:37455: BlockingRecv error: recv got EOF from 127.8.24.1:37455 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:29.750733 17471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:29.745803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:36497)
0620 07:07:29.746053 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:29.746065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:29.746150 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:29.746334 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:29.746336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:29.746507 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:29.746598 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.746603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.747281 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.747284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.747906 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.747915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.748272 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.748274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.748330 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.748836 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:29.748848 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:29.750582 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
W20260620 07:07:29.750819 17470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:29.745915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:36497 (local address 127.8.24.62:39915)
0620 07:07:29.746037 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:29.746040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:29.746119 (+ 79us) server_negotiation.cc:407] Connection header received
0620 07:07:29.746217 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:29.746219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:29.746259 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:29.746321 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:29.746675 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.747189 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.747987 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.748197 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.748262 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.750720 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:36497: BlockingRecv error: recv got EOF from 127.8.24.2:36497 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:29 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:29.930657 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:29.925601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:53837)
0620 07:07:29.925867 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:29.925883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:29.925991 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:29.926239 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:29.926242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:29.926429 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:29.926518 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.926524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.927337 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.927340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.928035 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:29.928043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.928438 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.928440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:29.928495 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.928922 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:29.928932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:29.930539 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":86,"threads_started":1}
W20260620 07:07:29.930788 17470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:29.925711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53837 (local address 127.8.24.62:39915)
0620 07:07:29.925867 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:29.925872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:29.925946 (+ 74us) server_negotiation.cc:407] Connection header received
0620 07:07:29.926094 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:29.926097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:29.926141 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:29.926204 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:29.926648 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.927218 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.928136 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:29.928355 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:29.928448 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:29.930674 (+ 2226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:53837: BlockingRecv error: recv got EOF from 127.8.24.3:53837 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:30.585007 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:30.580251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:39065)
0620 07:07:30.580504 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:30.580519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:30.580613 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:30.580822 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:30.580824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:30.580989 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:30.581088 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.581093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.581771 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.581774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.582324 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.582330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.582679 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.582682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.582737 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.583189 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:30.583199 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:30.584891 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
W20260620 07:07:30.585162 17474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:30.580331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39065 (local address 127.8.24.62:39915)
0620 07:07:30.580560 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:30.580563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:30.580575 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:30.580684 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:30.580686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:30.580726 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:30.580798 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:30.581168 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.581692 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.582396 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.582606 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.582670 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.585030 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:39065: BlockingRecv error: recv got EOF from 127.8.24.1:39065 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:30.756546 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:30.751600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:52421)
0620 07:07:30.751893 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:30.751908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:30.752019 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:30.752222 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:30.752224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:30.752396 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:30.752493 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.752498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.753262 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.753265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.753836 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.753846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.754238 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.754240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.754299 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.754770 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:30.754781 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:30.756429 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
W20260620 07:07:30.756675 17474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:30.751707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:52421 (local address 127.8.24.62:39915)
0620 07:07:30.751871 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:30.751874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:30.751982 (+ 108us) server_negotiation.cc:407] Connection header received
0620 07:07:30.752092 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:30.752094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:30.752139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:30.752209 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:30.752577 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.753170 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.753949 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.754159 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.754227 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.756571 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:52421: BlockingRecv error: recv got EOF from 127.8.24.2:52421 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:30 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:30.935922 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:30.931333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:46635)
0620 07:07:30.931596 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:30.931610 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:30.931700 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:30.931899 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:30.931901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:30.932065 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:30.932157 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.932161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.932814 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.932816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.933365 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:30.933371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.933742 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.933744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:30.933801 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.934230 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:30.934241 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:30.935814 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
W20260620 07:07:30.936058 17474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:30.931402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46635 (local address 127.8.24.62:39915)
0620 07:07:30.931542 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:30.931545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:30.931664 (+ 119us) server_negotiation.cc:407] Connection header received
0620 07:07:30.931767 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:30.931769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:30.931814 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:30.931876 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:30.932236 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.932741 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.933452 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:30.933664 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:30.933734 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:30.935948 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:46635: BlockingRecv error: recv got EOF from 127.8.24.3:46635 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:31.590691 17477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:31.585672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:50151)
0620 07:07:31.585999 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:31.586013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:31.586112 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:31.586324 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:31.586326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:31.586518 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:31.586613 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.586619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.587317 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.587320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.587970 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.587980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.588367 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.588369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.588444 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.588919 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:31.588931 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:31.590574 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":74,"threads_started":1}
W20260620 07:07:31.590824 17478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:31.585737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50151 (local address 127.8.24.62:39915)
0620 07:07:31.585994 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:31.585997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:31.586064 (+ 67us) server_negotiation.cc:407] Connection header received
0620 07:07:31.586189 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:31.586191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:31.586233 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:31.586296 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:31.586696 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.587231 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.588064 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.588288 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.588359 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.590694 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:50151: BlockingRecv error: recv got EOF from 127.8.24.1:50151 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:31.762377 17479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:31.757284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:45531)
0620 07:07:31.757605 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:31.757617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:31.757721 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:31.757942 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:31.757944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:31.758113 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:31.758205 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.758209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.758920 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.758923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.759525 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.759534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.759948 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.759950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.760009 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.760465 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:31.760477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:31.762219 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":96,"threads_started":1}
W20260620 07:07:31.762461 17478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:31.757449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45531 (local address 127.8.24.62:39915)
0620 07:07:31.757618 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:31.757621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:31.757684 (+ 63us) server_negotiation.cc:407] Connection header received
0620 07:07:31.757794 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:31.757796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:31.757837 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:31.757914 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:31.758294 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.758816 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.759632 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.759851 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.759937 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.762359 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:45531: BlockingRecv error: recv got EOF from 127.8.24.2:45531 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:31 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:31.941447 17480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:31.936619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:35589)
0620 07:07:31.936904 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:31.936925 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:31.937013 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:31.937211 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:31.937212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:31.937374 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:31.937464 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.937469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.938134 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.938136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.938696 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:31.938703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.939095 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.939097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:31.939153 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.939692 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:31.939704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:31.941330 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
W20260620 07:07:31.941597 17478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:31.936693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35589 (local address 127.8.24.62:39915)
0620 07:07:31.936809 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:31.936812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:31.936969 (+ 157us) server_negotiation.cc:407] Connection header received
0620 07:07:31.937086 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:31.937088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:31.937131 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:31.937185 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:31.937542 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.938050 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.938798 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:31.939015 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:31.939088 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:31.941467 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:35589: BlockingRecv error: recv got EOF from 127.8.24.3:35589 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:32.596527 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:32.591305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:46781)
0620 07:07:32.591624 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:32.591639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:32.591733 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:32.591980 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:32.591982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:32.592149 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:32.592241 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.592246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.593081 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.593084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.593658 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.593666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.594108 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.594110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.594166 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.594644 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:32.594655 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:32.596412 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
W20260620 07:07:32.596660 17482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:32.591520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:46781 (local address 127.8.24.62:39915)
0620 07:07:32.591749 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:32.591753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:32.591766 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:32.591800 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:32.591802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:32.591858 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:32.591936 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:32.592377 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.592974 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.593783 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.594020 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.594105 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.596542 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:46781: BlockingRecv error: recv got EOF from 127.8.24.1:46781 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:32.768591 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:32.763145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:49935)
0620 07:07:32.763386 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:32.763399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:32.763542 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:32.763787 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:32.763789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:32.763966 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:32.764062 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.764068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.764986 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.764989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.765561 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.765568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.765964 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.765966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.766019 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.766482 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:32.766492 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:32.768455 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
W20260620 07:07:32.768739 17482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:32.763229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:49935 (local address 127.8.24.62:39915)
0620 07:07:32.763399 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:32.763403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:32.763509 (+ 106us) server_negotiation.cc:407] Connection header received
0620 07:07:32.763631 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:32.763633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:32.763682 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:32.763739 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:32.764188 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.764888 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.765660 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.765879 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.765956 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.768623 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:49935: BlockingRecv error: recv got EOF from 127.8.24.2:49935 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:32 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:32.946858 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:32.942131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:43765)
0620 07:07:32.942388 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:32.942401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:32.942500 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:32.942710 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:32.942712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:32.942875 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:32.942976 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.942981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.943670 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.943672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.944231 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:32.944238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.944589 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.944591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:32.944647 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.945175 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:32.945187 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:32.946744 (+ 1557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
W20260620 07:07:32.947033 17482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:32.942228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:43765 (local address 127.8.24.62:39915)
0620 07:07:32.942379 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:32.942384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:32.942455 (+ 71us) server_negotiation.cc:407] Connection header received
0620 07:07:32.942577 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:32.942579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:32.942624 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:32.942679 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:32.943066 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.943597 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.944310 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:32.944514 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:32.944579 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:32.946879 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:43765: BlockingRecv error: recv got EOF from 127.8.24.3:43765 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:33.605535 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:33.597232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:48271)
0620 07:07:33.597505 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:33.597518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:33.597603 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:33.597854 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:33.597857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:33.598032 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:33.598123 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.598128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.598811 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.598814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.599363 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.599371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.599827 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.599830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.599899 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.603274 (+ 3375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:33.603286 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:33.605407 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
W20260620 07:07:33.605672 17486 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:33.597300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48271 (local address 127.8.24.62:39915)
0620 07:07:33.597564 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:33.597567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:33.597580 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:33.597690 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:33.597692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:33.597736 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:33.597810 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:33.598208 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.598720 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.599496 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.599726 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.599800 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.605541 (+ 5741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:48271: BlockingRecv error: recv got EOF from 127.8.24.1:48271 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:33.774484 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:33.769404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:55393)
0620 07:07:33.769677 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:33.769691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:33.769791 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:33.770009 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:33.770011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:33.770178 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:33.770270 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.770275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.770970 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.770972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.771568 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.771576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.771965 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.771967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.772021 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.772499 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:33.772510 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:33.774321 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":108,"threads_started":1}
W20260620 07:07:33.774573 17486 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:33.769503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:55393 (local address 127.8.24.62:39915)
0620 07:07:33.769625 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:33.769629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:33.769753 (+ 124us) server_negotiation.cc:407] Connection header received
0620 07:07:33.769870 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:33.769872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:33.769918 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:33.769974 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:33.770349 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.770884 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.771661 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.771875 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.771950 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.774473 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:55393: BlockingRecv error: recv got EOF from 127.8.24.2:55393 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:33 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:33.952512 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:33.947586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:45691)
0620 07:07:33.947831 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:33.947844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:33.947942 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:33.948132 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:33.948134 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:33.948297 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:33.948390 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.948395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.949055 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.949057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.949604 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:33.949610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.949999 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.950001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:33.950057 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.950552 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:33.950567 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:33.952376 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
W20260620 07:07:33.952720 17486 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:33.947615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45691 (local address 127.8.24.62:39915)
0620 07:07:33.947744 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:33.947748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:33.947906 (+ 158us) server_negotiation.cc:407] Connection header received
0620 07:07:33.948015 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:33.948017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:33.948066 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:33.948120 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:33.948472 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.948978 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.949700 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:33.949919 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:33.949993 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:33.952515 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:45691: BlockingRecv error: recv got EOF from 127.8.24.3:45691 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:34.611626 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:34.606286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:38491)
0620 07:07:34.606551 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:34.606564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:34.606649 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:34.606867 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:34.606868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:34.607055 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:34.607152 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.607157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.607894 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.607897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.608482 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.608491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.608950 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.608953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.609025 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.609523 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:34.609535 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:34.611498 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
W20260620 07:07:34.611732 17490 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:34.606417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38491 (local address 127.8.24.62:39915)
0620 07:07:34.606642 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:34.606646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:34.606660 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:07:34.606717 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:34.606720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:34.606761 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:34.606841 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:34.607240 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.607798 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.608584 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.608808 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.608895 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.611611 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:38491: BlockingRecv error: recv got EOF from 127.8.24.1:38491 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":68,"threads_started":1}
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:34.780119 17491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:34.775268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:37209)
0620 07:07:34.775537 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:34.775550 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:34.775643 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:34.775849 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:34.775851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:34.776024 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:34.776117 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.776122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.776790 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.776792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.777347 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.777353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.777707 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.777709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.777765 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.778217 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:34.778228 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:34.779973 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
W20260620 07:07:34.780215 17490 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:34.775310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37209 (local address 127.8.24.62:39915)
0620 07:07:34.775478 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:34.775483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:34.775642 (+ 159us) server_negotiation.cc:407] Connection header received
0620 07:07:34.775707 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:34.775709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:34.775757 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:34.775829 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:34.776199 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.776710 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.777424 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.777632 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.777698 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.780115 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:37209: BlockingRecv error: recv got EOF from 127.8.24.2:37209 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:34 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:34.958076 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:34.953234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:43753)
0620 07:07:34.953488 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:34.953502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:34.953612 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:34.953863 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:34.953866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:34.954038 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:34.954135 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.954141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.954841 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.954844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.955423 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:34.955451 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.955806 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.955808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:34.955866 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.956280 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:34.956290 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:34.957961 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
W20260620 07:07:34.958189 17490 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:34.953341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:43753 (local address 127.8.24.62:39915)
0620 07:07:34.953519 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:34.953525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:34.953557 (+ 32us) server_negotiation.cc:407] Connection header received
0620 07:07:34.953696 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:34.953698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:34.953748 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:34.953820 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:34.954223 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.954750 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.955527 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:34.955733 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:34.955799 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:34.958087 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:43753: BlockingRecv error: recv got EOF from 127.8.24.3:43753 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:35.617416 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:35.612337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:57591)
0620 07:07:35.612628 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:35.612644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:35.612746 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:35.613031 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:35.613033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:35.613254 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:35.613377 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.613384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.614106 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.614108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.614726 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.614735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.615130 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.615133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.615192 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.615683 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:35.615695 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:35.617283 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
W20260620 07:07:35.617547 17494 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:35.612468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57591 (local address 127.8.24.62:39915)
0620 07:07:35.612674 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:35.612679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:35.612694 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:07:35.612839 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:35.612842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:35.612905 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:35.612994 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:35.613480 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.614026 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.614823 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.615053 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.615122 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.617416 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:57591: BlockingRecv error: recv got EOF from 127.8.24.1:57591 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":58,"threads_started":1}
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:35.785851 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:35.780808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:48603)
0620 07:07:35.781054 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:35.781067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:35.781164 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:35.781367 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:35.781369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:35.781548 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:35.781644 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.781649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.782339 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.782342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.783052 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.783065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.783607 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.783610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.783674 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.784105 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:35.784116 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:35.785720 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":67,"threads_started":1}
W20260620 07:07:35.785957 17494 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:35.780893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48603 (local address 127.8.24.62:39915)
0620 07:07:35.781031 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:35.781035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:35.781127 (+ 92us) server_negotiation.cc:407] Connection header received
0620 07:07:35.781233 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:35.781235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:35.781277 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:35.781342 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:35.781740 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.782257 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.783185 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.783510 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.783591 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.785845 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:48603: BlockingRecv error: recv got EOF from 127.8.24.2:48603 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:35 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:35.963900 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:35.958833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:33671)
0620 07:07:35.959095 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:35.959106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:35.959210 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:35.959475 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:35.959478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:35.959672 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:35.959760 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.959766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.960491 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.960495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.961053 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:35.961061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.961486 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.961490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:35.961566 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.962041 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:35.962055 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:35.963723 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
W20260620 07:07:35.963963 17494 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:35.958907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:33671 (local address 127.8.24.62:39915)
0620 07:07:35.959038 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:35.959042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:35.959170 (+ 128us) server_negotiation.cc:407] Connection header received
0620 07:07:35.959296 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:35.959299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:35.959346 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:35.959459 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:35.959868 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.960393 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.961162 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:35.961391 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:35.961474 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:35.963862 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:33671: BlockingRecv error: recv got EOF from 127.8.24.3:33671 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:36.622895 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:36.618141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:39631)
0620 07:07:36.618404 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:36.618416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:36.618512 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:36.618733 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:36.618735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:36.618907 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:36.619003 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.619008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.619715 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.619717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.620262 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.620269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.620679 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.620681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.620735 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.621191 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:36.621201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:36.622778 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
W20260620 07:07:36.623035 17498 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:36.618220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39631 (local address 127.8.24.62:39915)
0620 07:07:36.618477 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:36.618481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:36.618494 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:36.618589 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:36.618591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:36.618640 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:36.618710 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:36.619087 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.619644 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.620358 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.620598 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.620670 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.622913 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:39631: BlockingRecv error: recv got EOF from 127.8.24.1:39631 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":57,"threads_started":1}
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:36.791569 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:36.786501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:43705)
0620 07:07:36.786746 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:36.786759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:36.786859 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:36.787090 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:36.787092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:36.787266 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:36.787362 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.787367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.788157 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.788160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.788725 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.788733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.789124 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.789126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.789182 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.789629 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:36.789640 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:36.791421 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":68,"threads_started":1}
W20260620 07:07:36.791698 17498 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:36.786594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:43705 (local address 127.8.24.62:39915)
0620 07:07:36.786750 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:36.786754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:36.786820 (+ 66us) server_negotiation.cc:407] Connection header received
0620 07:07:36.786941 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:36.786943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:36.786997 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:36.787060 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:36.787500 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.788044 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.788830 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.789043 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.789118 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.791588 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:43705: BlockingRecv error: recv got EOF from 127.8.24.2:43705 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:36 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:36.969568 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:36.964728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:53273)
0620 07:07:36.964990 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:36.965003 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:36.965106 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:36.965345 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:36.965347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:36.965520 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:36.965615 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.965620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.966290 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.966293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.966872 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:36.966887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.967247 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.967249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:36.967309 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.967789 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:36.967800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:36.969455 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":72,"threads_started":1}
W20260620 07:07:36.969717 17498 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:36.964785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53273 (local address 127.8.24.62:39915)
0620 07:07:36.965029 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:36.965033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:36.965055 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:07:36.965202 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:36.965205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:36.965256 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:36.965319 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:36.965700 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.966208 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.966963 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:36.967172 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:36.967239 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:36.969594 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:53273: BlockingRecv error: recv got EOF from 127.8.24.3:53273 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:37.628616 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:37.623604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:50005)
0620 07:07:37.623865 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:37.623879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:37.623967 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:37.624180 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:37.624182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:37.624348 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:37.624441 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.624446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.625163 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.625166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.625746 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.625754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.626210 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.626212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.626270 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.626745 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:37.626755 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:37.628479 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
W20260620 07:07:37.628750 17502 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:37.623676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:50005 (local address 127.8.24.62:39915)
0620 07:07:37.623913 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:37.623917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:37.623929 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:07:37.624040 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:37.624042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:37.624084 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:37.624155 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:37.624541 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.625071 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.625879 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.626111 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.626192 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.628617 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:50005: BlockingRecv error: recv got EOF from 127.8.24.1:50005 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:37.797168 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:37.792259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:41845)
0620 07:07:37.792523 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:37.792539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:37.792636 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:37.792897 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:37.792899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:37.793101 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:37.793218 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.793226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.793931 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.793933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.794511 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.794519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.794879 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.794882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.794938 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.795404 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:37.795419 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:37.797055 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
W20260620 07:07:37.797286 17502 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:37.792381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:41845 (local address 127.8.24.62:39915)
0620 07:07:37.792572 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:37.792577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:37.792591 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:07:37.792727 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:37.792730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:37.792783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:37.792861 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:37.793324 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.793844 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.794594 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.794799 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.794868 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.797170 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:41845: BlockingRecv error: recv got EOF from 127.8.24.2:41845 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:37 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:37.976027 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:37.970278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:60019)
0620 07:07:37.970597 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:37.970611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:37.970727 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:37.970973 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:37.970976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:37.971171 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:37.971275 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.971283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.972066 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.972069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.972710 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:37.972720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.973227 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.973231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:37.973335 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.973920 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:37.973934 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:37.975878 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":123,"threads_started":1}
W20260620 07:07:37.976125 17502 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:37.970406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60019 (local address 127.8.24.62:39915)
0620 07:07:37.970566 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:37.970571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:37.970684 (+ 113us) server_negotiation.cc:407] Connection header received
0620 07:07:37.970819 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:37.970822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:37.970871 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:37.970935 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:37.971393 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.971952 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.972821 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:37.973109 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:37.973208 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:37.976025 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:60019: BlockingRecv error: recv got EOF from 127.8.24.3:60019 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:38.637544 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:38.629281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:36431)
0620 07:07:38.629587 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:38.629604 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:38.629701 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:38.629961 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:38.629963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:38.630133 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:38.630232 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.630237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.630922 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.630925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.631545 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.631554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.631909 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.631912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.631967 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.635325 (+ 3358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:38.635338 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:38.637375 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":77,"threads_started":1}
W20260620 07:07:38.637691 17506 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:38.629393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:36431 (local address 127.8.24.62:39915)
0620 07:07:38.629688 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:38.629692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:38.629705 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:38.629769 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:38.629771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:38.629813 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:38.629885 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:38.630321 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.630838 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.631628 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.631835 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.631900 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.637567 (+ 5667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:36431: BlockingRecv error: recv got EOF from 127.8.24.1:36431 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":76,"threads_started":1}
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:38.803316 17507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:38.797922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:55257)
0620 07:07:38.798212 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:38.798224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:38.798333 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:38.798584 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:38.798587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:38.798773 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:38.798866 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.798872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.799659 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.799663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.800275 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.800286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.800877 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.800881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.800964 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.801519 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:38.801535 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:38.803188 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
W20260620 07:07:38.803627 17506 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:38.798003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:55257 (local address 127.8.24.62:39915)
0620 07:07:38.798159 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:38.798164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:38.798292 (+ 128us) server_negotiation.cc:407] Connection header received
0620 07:07:38.798421 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:38.798424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:38.798478 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:38.798544 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:38.798982 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.799560 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.800451 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.800730 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.800840 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.803322 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:55257: BlockingRecv error: recv got EOF from 127.8.24.2:55257 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:38 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.3@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:38.981628 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:38.976699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.3:57415)
0620 07:07:38.976976 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:38.976988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:38.977092 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:38.977325 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:38.977328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:38.977513 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:38.977605 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.977610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.978323 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.978326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.978875 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:38.978882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.979297 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.979301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:38.979378 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.979838 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:38.979850 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:38.981506 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
W20260620 07:07:38.981739 17506 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:38.976765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:57415 (local address 127.8.24.62:39915)
0620 07:07:38.976903 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:38.976908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:38.977052 (+ 144us) server_negotiation.cc:407] Connection header received
0620 07:07:38.977175 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:38.977178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:38.977227 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:38.977289 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:38.977713 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.978227 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.978984 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:38.979204 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:38.979287 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:38.981637 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.3:57415: BlockingRecv error: recv got EOF from 127.8.24.3:57415 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 20 07:07:39 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:39 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.1@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:39.643620 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:39.638442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.1:51919)
0620 07:07:39.638725 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:39.638738 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:39.638819 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:39.639033 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:39.639035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:39.639198 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:39.639288 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:39.639293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.640004 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.640006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:39.640715 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:39.640725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.641223 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.641227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:39.641312 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:39.641794 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:39.641806 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:39.643490 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
W20260620 07:07:39.643704 17510 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:39.638529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:51919 (local address 127.8.24.62:39915)
0620 07:07:39.638765 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:39.638768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:39.638781 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:07:39.638891 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:39.638893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:39.638935 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:39.639005 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:39.639383 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.639932 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.640825 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.641115 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.641196 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:39.643595 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.1:51919: BlockingRecv error: recv got EOF from 127.8.24.1:51919 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":59,"threads_started":1}
Jun 20 07:07:39 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 07:07:39 dist-test-slave-wjw3 krb5kdc[16451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.24.2@KRBTEST.COM for oryx/127.8.24.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 07:07:39.809291 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:39.804131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39915 (local address 127.8.24.2:59435)
0620 07:07:39.804383 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:39.804397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:39.804505 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:39.804738 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:39.804741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:39.804942 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:39.805044 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:39.805050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.805795 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.805799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:39.806378 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:39.806386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.806814 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.806817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:39.806899 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:39.807349 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:39.807360 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:39.809131 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.24.62:39915: Server oryx/127.8.24.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
W20260620 07:07:39.809365 17510 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:39.804214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:59435 (local address 127.8.24.62:39915)
0620 07:07:39.804356 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:39.804361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:39.804468 (+ 107us) server_negotiation.cc:407] Connection header received
0620 07:07:39.804593 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:39.804596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:39.804641 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:39.804704 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:39.805153 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.805683 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.806495 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:39.806716 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:39.806800 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:39.809264 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.24.2:59435: BlockingRecv error: recv got EOF from 127.8.24.2:59435 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:39.888530 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17034
I20260620 07:07:39.893240 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17186
I20260620 07:07:39.897535 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17319
I20260620 07:07:39.901378 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 16964
2026-06-20T07:07:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17714 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T07:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:39Z Disabled control of system clock
I20260620 07:07:39.928346 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42173
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:42173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 07:07:40.032272 17515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:40.032516 17515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:40.032547 17515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:40.036374 17515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:40.036419 17515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:40.036432 17515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:40.036448 17515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:40.036460 17515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:40.040979 17515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:42173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:42173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:40.041904 17515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:40.043201 17515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:07:40.051283 17515 server_base.cc:1034] running on GCE node
W20260620 07:07:40.052055 17521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.051999 17523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.052075 17520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.052739 17515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:40.053804 17515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:40.054980 17515 hybrid_clock.cc:648] HybridClock initialized: now 1781939260054959 us; error 32 us; skew 500 ppm
I20260620 07:07:40.056969 17515 webserver.cc:460] Webserver started at http://127.8.24.62:34863/ using document root <none> and password file <none>
I20260620 07:07:40.057627 17515 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:40.057713 17515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:40.057946 17515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:40.060263 17515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "8cc5c1e582fa47ceb34b54a060c02498"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "7e4cc20e0cdea7203716898177b8fc38"
server_key_iv: "71fa23e6ed3b6aa12ded4f0f08db243e"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.060765 17515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "8cc5c1e582fa47ceb34b54a060c02498"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "7e4cc20e0cdea7203716898177b8fc38"
server_key_iv: "71fa23e6ed3b6aa12ded4f0f08db243e"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.064175 17515 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:40.066418 17529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.067664 17515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:40.067762 17515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "8cc5c1e582fa47ceb34b54a060c02498"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "7e4cc20e0cdea7203716898177b8fc38"
server_key_iv: "71fa23e6ed3b6aa12ded4f0f08db243e"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.067868 17515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:40.076614 17515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:40.077172 17515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:40.077324 17515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:40.083549 17515 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:42173
I20260620 07:07:40.083570 17581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:42173 every 8 connection(s)
I20260620 07:07:40.084389 17515 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:40.086993 17582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:40.092204 17582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498: Bootstrap starting.
I20260620 07:07:40.092849 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17515
I20260620 07:07:40.092932 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:40.093194 8288 external_mini_cluster.cc:1428] Setting key 5466e82426f48d0a1d3ca3ab5d92d612
I20260620 07:07:40.094730 17582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:40.095395 17582 log.cc:824] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:40.097427 17582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498: No bootstrap required, opened a new log
I20260620 07:07:40.101475 17582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } }
I20260620 07:07:40.101701 17582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:40.101758 17582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc5c1e582fa47ceb34b54a060c02498, State: Initialized, Role: FOLLOWER
I20260620 07:07:40.102144 17582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } }
I20260620 07:07:40.102263 17582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:40.102324 17582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:40.102370 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.094351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42956 (local address 127.8.24.62:42173)
0620 07:07:40.094929 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.094943 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.094979 (+ 36us) server_negotiation.cc:407] Connection header received
0620 07:07:40.095648 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.095671 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.096151 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.096508 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.096825 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.097782 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.098953 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.099404 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.099574 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.100563 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:40.100570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:40.100589 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:40.101026 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:40.101080 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:40.101099 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:40.101173 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:40.101317 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 07:07:40.101564 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260620 07:07:40.102413 17582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:40.103374 17582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } }
I20260620 07:07:40.103706 17582 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc5c1e582fa47ceb34b54a060c02498; no voters:
I20260620 07:07:40.103997 17582 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:40.104092 17587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:40.104305 17587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [term 1 LEADER]: Becoming Leader. State: Replica: 8cc5c1e582fa47ceb34b54a060c02498, State: Running, Role: LEADER
I20260620 07:07:40.104578 17587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } }
I20260620 07:07:40.105087 17582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:40.105556 17587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cc5c1e582fa47ceb34b54a060c02498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } } }
I20260620 07:07:40.105664 17587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:40.105886 17587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc5c1e582fa47ceb34b54a060c02498. Latest consensus state: current_term: 1 leader_uuid: "8cc5c1e582fa47ceb34b54a060c02498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc5c1e582fa47ceb34b54a060c02498" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 42173 } } }
I20260620 07:07:40.106032 17587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:40.106293 17594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:40.109145 17594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:40.114183 17594 catalog_manager.cc:1347] Generated new cluster ID: ff33c2bea9724c06bf3924633ed9666f
I20260620 07:07:40.114234 17594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:40.142554 17594 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:40.143505 17594 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:40.153048 17594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc5c1e582fa47ceb34b54a060c02498: Generated new TSK 0
I20260620 07:07:40.153664 17594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:40.158020 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--builtin_ntp_servers=127.8.24.20:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 07:07:40.270486 17606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:40.270725 17606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:40.270761 17606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:40.274729 17606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:40.274773 17606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:40.274829 17606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:40.279314 17606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:40.280298 17606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:40.281558 17606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:40.290225 17614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.290208 17611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.290256 17612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.290307 17606 server_base.cc:1034] running on GCE node
I20260620 07:07:40.290777 17606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:40.291379 17606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:40.292548 17606 hybrid_clock.cc:648] HybridClock initialized: now 1781939260292530 us; error 30 us; skew 500 ppm
I20260620 07:07:40.294253 17606 webserver.cc:460] Webserver started at http://127.8.24.1:43831/ using document root <none> and password file <none>
I20260620 07:07:40.294834 17606 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:40.294900 17606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:40.295082 17606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:40.297091 17606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "d1d7e4930f754512a20e3191b7089b3d"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "c69a6cb89e4c23f154c61da9456646d2"
server_key_iv: "afbe50bc19b8d16b1980f5faf55b6a82"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.297613 17606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "d1d7e4930f754512a20e3191b7089b3d"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "c69a6cb89e4c23f154c61da9456646d2"
server_key_iv: "afbe50bc19b8d16b1980f5faf55b6a82"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.301051 17606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 07:07:40.303325 17620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.304514 17606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:40.304615 17606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "d1d7e4930f754512a20e3191b7089b3d"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "c69a6cb89e4c23f154c61da9456646d2"
server_key_iv: "afbe50bc19b8d16b1980f5faf55b6a82"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.304705 17606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:40.317245 17606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:40.317916 17606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:40.318086 17606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:40.318650 17606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:40.319568 17606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:40.319613 17606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.319665 17606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:40.319693 17606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.327692 17606 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44995
I20260620 07:07:40.327754 17733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44995 every 8 connection(s)
I20260620 07:07:40.328559 17606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:40.334126 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17606
I20260620 07:07:40.334230 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:40.334535 8288 external_mini_cluster.cc:1428] Setting key ecb04692b46609db7eec37836f4c6cf8
I20260620 07:07:40.336400 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--builtin_ntp_servers=127.8.24.20:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 07:07:40.337517 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.330257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34943 (local address 127.8.24.62:42173)
0620 07:07:40.330394 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.330397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.331378 (+ 981us) server_negotiation.cc:407] Connection header received
0620 07:07:40.332753 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.332756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.332802 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.332876 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.333722 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.334495 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.335723 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.335985 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.336088 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.337088 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:40.337092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:40.337094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:40.337107 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:40.337128 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:40.337137 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:40.337202 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:40.337314 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 07:07:40.337375 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:07:40.338490 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.330640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42173 (local address 127.8.24.1:34943)
0620 07:07:40.331261 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:40.331299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:40.332559 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:40.333032 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:40.333039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:40.333118 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:40.333581 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.333592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.334618 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.334621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.335615 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.335624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.336125 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.336128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.336268 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.336839 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:40.336840 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:40.336878 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:40.336937 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:40.337212 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:40.337217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:40.337221 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:40.337300 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 07:07:40.337528 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260620 07:07:40.339603 17734 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42173
I20260620 07:07:40.339834 17734 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:40.340456 17734 heartbeater.cc:511] Master 127.8.24.62:42173 requested a full tablet report, sending...
I20260620 07:07:40.342092 17546 ts_manager.cc:194] Registered new tserver with Master: d1d7e4930f754512a20e3191b7089b3d (127.8.24.1:44995)
I20260620 07:07:40.343672 17546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:34943
W20260620 07:07:40.443389 17737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:40.443650 17737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:40.443684 17737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:40.447575 17737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:40.447618 17737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:40.447667 17737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:40.452246 17737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:40.453181 17737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:40.454444 17737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:40.463253 17743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.463244 17745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.463289 17742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.463574 17737 server_base.cc:1034] running on GCE node
I20260620 07:07:40.463905 17737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:40.464478 17737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:40.465633 17737 hybrid_clock.cc:648] HybridClock initialized: now 1781939260465616 us; error 32 us; skew 500 ppm
I20260620 07:07:40.467342 17737 webserver.cc:460] Webserver started at http://127.8.24.2:33709/ using document root <none> and password file <none>
I20260620 07:07:40.467905 17737 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:40.467971 17737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:40.468145 17737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:40.470065 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "3c3bdf5727ed4ce4a62ef64fde088ae3"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "943a641328804eaa5537f357909df72a"
server_key_iv: "60379a4b0fff820e9c32b5cca070e12f"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.470575 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "3c3bdf5727ed4ce4a62ef64fde088ae3"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "943a641328804eaa5537f357909df72a"
server_key_iv: "60379a4b0fff820e9c32b5cca070e12f"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.474119 17737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:07:40.476604 17751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.477751 17737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:07:40.477860 17737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "3c3bdf5727ed4ce4a62ef64fde088ae3"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "943a641328804eaa5537f357909df72a"
server_key_iv: "60379a4b0fff820e9c32b5cca070e12f"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.477955 17737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:40.507550 17737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:40.508272 17737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:40.508441 17737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:40.509013 17737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:40.509941 17737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:40.509995 17737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.510038 17737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:40.510066 17737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.518071 17737 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40713
I20260620 07:07:40.518123 17864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40713 every 8 connection(s)
I20260620 07:07:40.518980 17737 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:40.522526 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17737
I20260620 07:07:40.522627 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:40.522879 8288 external_mini_cluster.cc:1428] Setting key be104e3902aa64807f1dd97dbab7dd00
I20260620 07:07:40.524605 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--builtin_ntp_servers=127.8.24.20:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 07:07:40.528066 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.520717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:33969 (local address 127.8.24.62:42173)
0620 07:07:40.520852 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.520856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.521905 (+ 1049us) server_negotiation.cc:407] Connection header received
0620 07:07:40.523299 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.523303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.523357 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.523448 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.524329 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.524884 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.526243 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.526458 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.526537 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.527621 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:40.527624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:40.527627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:40.527640 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:40.527661 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:40.527669 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:40.527727 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:40.527861 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 07:07:40.527919 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 07:07:40.528903 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.521131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42173 (local address 127.8.24.2:33969)
0620 07:07:40.521764 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:40.521805 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:40.523103 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:40.523631 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:40.523638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:40.523724 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:40.524201 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.524213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.525001 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.525004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.526113 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.526124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.526600 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.526604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.526777 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.527347 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:40.527349 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:40.527388 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:40.527489 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:40.527735 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:40.527743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:40.527749 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:07:40.527856 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 07:07:40.528133 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
I20260620 07:07:40.529963 17865 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42173
I20260620 07:07:40.530185 17865 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:40.530692 17865 heartbeater.cc:511] Master 127.8.24.62:42173 requested a full tablet report, sending...
I20260620 07:07:40.531715 17546 ts_manager.cc:194] Registered new tserver with Master: 3c3bdf5727ed4ce4a62ef64fde088ae3 (127.8.24.2:40713)
I20260620 07:07:40.532472 17546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:33969
W20260620 07:07:40.631901 17868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:40.632128 17868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:40.632161 17868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:40.636020 17868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:40.636066 17868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:40.636116 17868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:40.640673 17868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:40.641629 17868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:40.642904 17868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:40.652186 17876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.652281 17873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.652215 17874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.652331 17868 server_base.cc:1034] running on GCE node
I20260620 07:07:40.652853 17868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:40.653439 17868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:40.654577 17868 hybrid_clock.cc:648] HybridClock initialized: now 1781939260654555 us; error 37 us; skew 500 ppm
I20260620 07:07:40.656302 17868 webserver.cc:460] Webserver started at http://127.8.24.3:37647/ using document root <none> and password file <none>
I20260620 07:07:40.656981 17868 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:40.657051 17868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:40.657240 17868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:40.659181 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "a98ecc0c9af04080b27d33d593b31f10"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "e8444de5d2b2bf72da5ac6290e7d26a5"
server_key_iv: "77a33992988cd3ef0b7ea03655c93a91"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.659819 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "a98ecc0c9af04080b27d33d593b31f10"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "e8444de5d2b2bf72da5ac6290e7d26a5"
server_key_iv: "77a33992988cd3ef0b7ea03655c93a91"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.663239 17868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:40.665643 17882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.666786 17868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:40.666886 17868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "a98ecc0c9af04080b27d33d593b31f10"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "e8444de5d2b2bf72da5ac6290e7d26a5"
server_key_iv: "77a33992988cd3ef0b7ea03655c93a91"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.666977 17868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:40.684052 17868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:40.684691 17868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:40.684847 17868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:40.685388 17868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:40.686275 17868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:40.686322 17868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.686372 17868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:40.686398 17868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.694455 17868 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:35885
I20260620 07:07:40.694475 17995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:35885 every 8 connection(s)
I20260620 07:07:40.695279 17868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:40.700476 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 17868
I20260620 07:07:40.700564 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:40.700785 8288 external_mini_cluster.cc:1428] Setting key c26e67cff8989558f070ec0324570c8f
I20260620 07:07:40.703761 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.696960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:45897 (local address 127.8.24.62:42173)
0620 07:07:40.697098 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.697103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.698112 (+ 1009us) server_negotiation.cc:407] Connection header received
0620 07:07:40.699354 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.699357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.699401 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.699498 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.700296 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.700865 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.702121 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.702353 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.702427 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.703386 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:40.703389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:40.703391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:40.703402 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:40.703421 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:40.703428 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:40.703509 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:40.703618 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 07:07:40.703660 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:40.704528 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.697374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:42173 (local address 127.8.24.3:45897)
0620 07:07:40.697968 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:40.698008 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:40.699183 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:40.699643 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:40.699650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:40.699730 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:40.700174 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.700185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.700978 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.700982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.702014 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:40.702025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:40.702490 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:40.702494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:40.702649 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:40.703159 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:40.703160 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:40.703198 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:40.703254 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:40.703510 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:40.703523 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:40.703527 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:40.703614 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 07:07:40.703813 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260620 07:07:40.705461 17996 heartbeater.cc:347] Connected to a master server at 127.8.24.62:42173
I20260620 07:07:40.705678 17996 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:40.706171 17996 heartbeater.cc:511] Master 127.8.24.62:42173 requested a full tablet report, sending...
I20260620 07:07:40.707058 17546 ts_manager.cc:194] Registered new tserver with Master: a98ecc0c9af04080b27d33d593b31f10 (127.8.24.3:35885)
I20260620 07:07:40.707620 17546 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:45897
I20260620 07:07:40.714145 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 07:07:40.716810 17585 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:40.715926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.8.24.62:42173)
0620 07:07:40.716041 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.716046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.716219 (+ 173us) server_negotiation.cc:407] Connection header received
0620 07:07:40.716330 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.716333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.716375 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.716437 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.716687 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42972: BlockingRecv error: recv got EOF from 127.0.0.1:42972 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:40.716930 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17606
I20260620 07:07:40.722160 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17737
I20260620 07:07:40.726203 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17868
I20260620 07:07:40.730206 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 17515
2026-06-20T07:07:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (828 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T07:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:40Z Disabled control of system clock
I20260620 07:07:40.757107 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45187
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 07:07:40.860697 18007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:40.860927 18007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:40.860975 18007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:40.864833 18007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:40.864887 18007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:40.864907 18007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:40.864926 18007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:40.864945 18007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:40.869520 18007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:40.870487 18007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:40.871791 18007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:40.879426 18012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:40.879462 18013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.879557 18007 server_base.cc:1034] running on GCE node
W20260620 07:07:40.879490 18015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:40.880308 18007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:40.881381 18007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:40.882527 18007 hybrid_clock.cc:648] HybridClock initialized: now 1781939260882506 us; error 41 us; skew 500 ppm
I20260620 07:07:40.884296 18007 webserver.cc:460] Webserver started at http://127.8.24.62:43257/ using document root <none> and password file <none>
I20260620 07:07:40.884825 18007 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:40.884892 18007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:40.885090 18007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:40.887076 18007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "3b87464c57644cc5ae79fb6e3a4b34e2"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "06ed51c42c8d92fb2bbf8e0bbc672222"
server_key_iv: "f23a2108e63c4505b9e647fa77f4d93c"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.887640 18007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "3b87464c57644cc5ae79fb6e3a4b34e2"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "06ed51c42c8d92fb2bbf8e0bbc672222"
server_key_iv: "f23a2108e63c4505b9e647fa77f4d93c"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.891108 18007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260620 07:07:40.893391 18021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:40.894531 18007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:40.894636 18007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "3b87464c57644cc5ae79fb6e3a4b34e2"
format_stamp: "Formatted at 2026-06-20 07:07:40 on dist-test-slave-wjw3"
server_key: "06ed51c42c8d92fb2bbf8e0bbc672222"
server_key_iv: "f23a2108e63c4505b9e647fa77f4d93c"
server_key_version: "encryptionkey@0"
I20260620 07:07:40.894726 18007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:40.898918 18007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:40.899482 18007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:40.899622 18007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:40.905674 18007 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45187
I20260620 07:07:40.905699 18073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45187 every 8 connection(s)
I20260620 07:07:40.906497 18007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:40.909408 18074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:40.911347 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18007
I20260620 07:07:40.911484 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:40.911728 8288 external_mini_cluster.cc:1428] Setting key 2cc77bee06a7b8d10195a421964d0808
I20260620 07:07:40.915190 18074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2: Bootstrap starting.
I20260620 07:07:40.917333 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:40.913027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.8.24.62:45187)
0620 07:07:40.913658 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:40.913677 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:40.913717 (+ 40us) server_negotiation.cc:407] Connection header received
0620 07:07:40.914511 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:40.914534 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:40.914953 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:40.915313 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:40.915967 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:40.915972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:40.915985 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:40.915996 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:40.916052 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:40.916074 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:40.916202 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:40.916271 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 07:07:40.916520 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260620 07:07:40.917685 18074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:40.918485 18074 log.cc:824] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:40.920293 18074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2: No bootstrap required, opened a new log
I20260620 07:07:40.923460 18074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } }
I20260620 07:07:40.923669 18074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:40.923714 18074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b87464c57644cc5ae79fb6e3a4b34e2, State: Initialized, Role: FOLLOWER
I20260620 07:07:40.924134 18074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } }
I20260620 07:07:40.924279 18074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:40.924350 18074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:40.924444 18074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:40.925212 18074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } }
I20260620 07:07:40.925508 18074 leader_election.cc:302] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b87464c57644cc5ae79fb6e3a4b34e2; no voters:
I20260620 07:07:40.925755 18074 leader_election.cc:288] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:40.925838 18079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:40.926031 18079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b87464c57644cc5ae79fb6e3a4b34e2, State: Running, Role: LEADER
I20260620 07:07:40.926311 18079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } }
I20260620 07:07:40.926797 18074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:40.927807 18080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } } }
I20260620 07:07:40.927934 18080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:40.928184 18081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b87464c57644cc5ae79fb6e3a4b34e2. Latest consensus state: current_term: 1 leader_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87464c57644cc5ae79fb6e3a4b34e2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45187 } } }
I20260620 07:07:40.928270 18081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:40.928287 18088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:40.930905 18088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:40.935854 18088 catalog_manager.cc:1347] Generated new cluster ID: 5eb85658fbd54c298174b28d4c6579bc
I20260620 07:07:40.935904 18088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:40.947549 18088 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:40.947824 18088 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:40.951112 18088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b87464c57644cc5ae79fb6e3a4b34e2: Generated new TSK 0
I20260620 07:07:40.951720 18088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:07:40.955755 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--builtin_ntp_servers=127.8.24.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 07:07:41.061896 18098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:41.062147 18098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:41.062184 18098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:41.066111 18098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:41.066156 18098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:41.066210 18098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:41.070746 18098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:41.071718 18098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:41.073000 18098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:41.082050 18104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:41.082059 18103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:41.082118 18098 server_base.cc:1034] running on GCE node
W20260620 07:07:41.082074 18106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:41.082626 18098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:41.083220 18098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:41.084380 18098 hybrid_clock.cc:648] HybridClock initialized: now 1781939261084345 us; error 47 us; skew 500 ppm
I20260620 07:07:41.085997 18098 webserver.cc:460] Webserver started at http://127.8.24.1:38329/ using document root <none> and password file <none>
I20260620 07:07:41.086548 18098 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:41.086611 18098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:41.086786 18098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:41.088739 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "5922e1fce47a4aa4b69843a99ae06935"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "2a3014368f193118ce074a3f2e846392"
server_key_iv: "90877431e7ec1c0415a7d2b4f9f9949a"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.089258 18098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "5922e1fce47a4aa4b69843a99ae06935"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "2a3014368f193118ce074a3f2e846392"
server_key_iv: "90877431e7ec1c0415a7d2b4f9f9949a"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.095242 18098 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260620 07:07:41.098335 18112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.099880 18098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:07:41.099985 18098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "5922e1fce47a4aa4b69843a99ae06935"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "2a3014368f193118ce074a3f2e846392"
server_key_iv: "90877431e7ec1c0415a7d2b4f9f9949a"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.100091 18098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:41.105841 18098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:41.106501 18098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:41.106664 18098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:41.107235 18098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:41.108143 18098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:41.108188 18098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.108235 18098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:41.108263 18098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.116060 18098 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:39925
I20260620 07:07:41.116082 18225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:39925 every 8 connection(s)
I20260620 07:07:41.116842 18098 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:41.120836 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18098
I20260620 07:07:41.120934 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:41.121228 8288 external_mini_cluster.cc:1428] Setting key 001a3e1ca5331b32e42d601504ae49b8
I20260620 07:07:41.122638 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.118450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:44475 (local address 127.8.24.62:45187)
0620 07:07:41.118578 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:41.118582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:41.119641 (+ 1059us) server_negotiation.cc:407] Connection header received
0620 07:07:41.120887 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:41.120890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:41.120934 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:41.121012 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:41.122222 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:41.122225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:41.122228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:41.122240 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:41.122262 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:41.122271 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:41.122360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:41.122469 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 07:07:41.122512 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:07:41.123209 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--builtin_ntp_servers=127.8.24.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 07:07:41.123672 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.118812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45187 (local address 127.8.24.1:44475)
0620 07:07:41.119511 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:41.119551 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:41.120687 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:41.121223 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:41.121233 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:41.121346 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:41.121973 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:41.121975 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:41.122014 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:41.122082 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:41.122346 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:41.122350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:41.122354 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:41.122507 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 07:07:41.122731 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260620 07:07:41.124797 18226 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45187
I20260620 07:07:41.125031 18226 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:41.125398 18226 heartbeater.cc:511] Master 127.8.24.62:45187 requested a full tablet report, sending...
I20260620 07:07:41.126776 18038 ts_manager.cc:194] Registered new tserver with Master: 5922e1fce47a4aa4b69843a99ae06935 (127.8.24.1:39925)
W20260620 07:07:41.228444 18229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:41.228685 18229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:41.228736 18229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:41.232643 18229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:41.232692 18229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:41.232766 18229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:41.237323 18229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:41.238322 18229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:41.239651 18229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:41.248369 18235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:41.248368 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:41.248512 18229 server_base.cc:1034] running on GCE node
W20260620 07:07:41.248595 18237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:41.248965 18229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:41.249541 18229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:41.250690 18229 hybrid_clock.cc:648] HybridClock initialized: now 1781939261250665 us; error 39 us; skew 500 ppm
I20260620 07:07:41.252341 18229 webserver.cc:460] Webserver started at http://127.8.24.2:42171/ using document root <none> and password file <none>
I20260620 07:07:41.252872 18229 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:41.252934 18229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:41.253127 18229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:41.255039 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "4e25bc5a05aa4fa68ed961f19ee8b726"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "e6e8829c78eaffd29b8d8c1349779c45"
server_key_iv: "d9fb5bb77d771995b53fd014d9b61302"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.255592 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "4e25bc5a05aa4fa68ed961f19ee8b726"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "e6e8829c78eaffd29b8d8c1349779c45"
server_key_iv: "d9fb5bb77d771995b53fd014d9b61302"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.258934 18229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:07:41.261227 18243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.262320 18229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:41.262418 18229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "4e25bc5a05aa4fa68ed961f19ee8b726"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "e6e8829c78eaffd29b8d8c1349779c45"
server_key_iv: "d9fb5bb77d771995b53fd014d9b61302"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.262511 18229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:41.266690 18229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:41.267232 18229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:41.267373 18229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:41.267876 18229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:41.268693 18229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:41.268738 18229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.268785 18229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:41.268813 18229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.276798 18229 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:41823
I20260620 07:07:41.276815 18356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:41823 every 8 connection(s)
I20260620 07:07:41.277642 18229 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:41.278970 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18229
I20260620 07:07:41.279075 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:41.279322 8288 external_mini_cluster.cc:1428] Setting key ccc2a8b652c0d5f8b1a7a639635db66f
I20260620 07:07:41.281124 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--builtin_ntp_servers=127.8.24.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 07:07:41.283533 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.279486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37845 (local address 127.8.24.62:45187)
0620 07:07:41.279635 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:41.279640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:41.280737 (+ 1097us) server_negotiation.cc:407] Connection header received
0620 07:07:41.282009 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:41.282012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:41.282050 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:41.282134 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:41.283101 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:41.283105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:41.283108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:41.283120 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:41.283143 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:41.283153 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:41.283240 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:41.283329 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 07:07:41.283369 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:07:41.284307 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.279892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45187 (local address 127.8.24.2:37845)
0620 07:07:41.280595 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:41.280645 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:41.281809 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:41.282255 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:41.282261 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:41.282346 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:41.282849 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:41.282851 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:41.282889 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:41.282957 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:41.283226 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:41.283230 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:41.283234 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:07:41.283369 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 07:07:41.283586 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260620 07:07:41.285329 18357 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45187
I20260620 07:07:41.285528 18357 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:41.285884 18357 heartbeater.cc:511] Master 127.8.24.62:45187 requested a full tablet report, sending...
I20260620 07:07:41.286784 18038 ts_manager.cc:194] Registered new tserver with Master: 4e25bc5a05aa4fa68ed961f19ee8b726 (127.8.24.2:41823)
W20260620 07:07:41.387187 18360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:41.387418 18360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:41.387487 18360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:41.391326 18360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:41.391369 18360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:41.391419 18360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:41.395903 18360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:41.396852 18360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:41.398181 18360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:41.407090 18366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:41.407114 18368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:41.407096 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:41.407264 18360 server_base.cc:1034] running on GCE node
I20260620 07:07:41.407752 18360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:41.408313 18360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:41.409467 18360 hybrid_clock.cc:648] HybridClock initialized: now 1781939261409449 us; error 31 us; skew 500 ppm
I20260620 07:07:41.411124 18360 webserver.cc:460] Webserver started at http://127.8.24.3:40747/ using document root <none> and password file <none>
I20260620 07:07:41.411682 18360 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:41.411746 18360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:41.411932 18360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:41.413851 18360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "59910e1c5d344fb5b913a8c2b5ed5b2b"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "f469d9207477e9891f2de1893711a51d"
server_key_iv: "3635fb7625c1f679eccf91bbe8c37745"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.414390 18360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "59910e1c5d344fb5b913a8c2b5ed5b2b"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "f469d9207477e9891f2de1893711a51d"
server_key_iv: "3635fb7625c1f679eccf91bbe8c37745"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.417867 18360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:07:41.420266 18374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.421466 18360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260620 07:07:41.421566 18360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "59910e1c5d344fb5b913a8c2b5ed5b2b"
format_stamp: "Formatted at 2026-06-20 07:07:41 on dist-test-slave-wjw3"
server_key: "f469d9207477e9891f2de1893711a51d"
server_key_iv: "3635fb7625c1f679eccf91bbe8c37745"
server_key_version: "encryptionkey@0"
I20260620 07:07:41.421662 18360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:41.426062 18360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:41.426644 18360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:41.426815 18360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:41.427314 18360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:41.428288 18360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:41.428333 18360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.428380 18360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:41.428406 18360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:41.436213 18360 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46099
I20260620 07:07:41.436234 18487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46099 every 8 connection(s)
I20260620 07:07:41.437084 18360 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:41.442476 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.438732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:50489 (local address 127.8.24.62:45187)
0620 07:07:41.438874 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:41.438877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:41.439916 (+ 1039us) server_negotiation.cc:407] Connection header received
0620 07:07:41.441107 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:41.441110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:41.441145 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:41.441215 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:41.442138 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:41.442141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:41.442143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:07:41.442154 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:41.442173 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:07:41.442181 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:41.442248 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:41.442330 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 07:07:41.442365 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:07:41.443264 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:41.439108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45187 (local address 127.8.24.3:50489)
0620 07:07:41.439771 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:41.439811 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:41.440937 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:41.441361 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:41.441368 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:41.441455 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:41.441912 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:07:41.441914 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:41.441950 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:07:41.442014 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:41.442238 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:41.442242 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:41.442245 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:07:41.442361 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 07:07:41.442539 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":103,"threads_started":1}
I20260620 07:07:41.444275 18488 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45187
I20260620 07:07:41.444487 18488 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:41.444825 18488 heartbeater.cc:511] Master 127.8.24.62:45187 requested a full tablet report, sending...
I20260620 07:07:41.445757 18038 ts_manager.cc:194] Registered new tserver with Master: 59910e1c5d344fb5b913a8c2b5ed5b2b (127.8.24.3:46099)
I20260620 07:07:41.447129 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18360
I20260620 07:07:41.447242 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:07:41.447539 8288 external_mini_cluster.cc:1428] Setting key de43f30a5e5dc3a33507cba31d3b8f37
I20260620 07:07:41.449231 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 07:07:41.451740 18077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:07:41.450947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.8.24.62:45187)
0620 07:07:41.451060 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:41.451064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:41.451209 (+ 145us) server_negotiation.cc:407] Connection header received
0620 07:07:41.451321 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:41.451323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:41.451351 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:41.451411 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:41.451626 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38986: BlockingRecv error: recv got EOF from 127.0.0.1:38986 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 07:07:41.451853 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18098
I20260620 07:07:41.456876 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18229
I20260620 07:07:41.461261 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18360
I20260620 07:07:41.465521 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18007
2026-06-20T07:07:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (734 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:41 dist-test-slave-wjw3 krb5kdc[18498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:41 dist-test-slave-wjw3 krb5kdc[18498](info): set up 2 sockets
Jun 20 07:07:41 dist-test-slave-wjw3 krb5kdc[18498](info): commencing operation
krb5kdc: starting...
W20260620 07:07:42.412039 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:42 dist-test-slave-wjw3 krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:07:42.519204 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46597
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:42.624104 18511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:42.624351 18511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:42.624405 18511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:42.628289 18511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:07:42.628351 18511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:42.628377 18511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:42.628397 18511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:07:42.628418 18511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:07:42.633540 18511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:42.634557 18511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:42.635550 18511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:42.643492 18517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:42.643503 18516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:42.643513 18519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:42.643756 18511 server_base.cc:1034] running on GCE node
I20260620 07:07:42.644276 18511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:42.645288 18511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:42.646443 18511 hybrid_clock.cc:648] HybridClock initialized: now 1781939262646425 us; error 32 us; skew 500 ppm
Jun 20 07:07:42 dist-test-slave-wjw3 krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939262, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:42.648959 18511 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:07:42.649773 18511 webserver.cc:460] Webserver started at http://127.8.24.62:38225/ using document root <none> and password file <none>
I20260620 07:07:42.650293 18511 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:42.650357 18511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:42.650537 18511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:42.652551 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "4b7af8290547467e8b94fd5d739bdf23"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "6b5252acdd5a6953e9abfbbbe32c522f"
server_key_iv: "7587f6a59cf46ef545cd90a86b483517"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.653086 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "4b7af8290547467e8b94fd5d739bdf23"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "6b5252acdd5a6953e9abfbbbe32c522f"
server_key_iv: "7587f6a59cf46ef545cd90a86b483517"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.656641 18511 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260620 07:07:42.658816 18526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:42.660012 18511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:07:42.660117 18511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "4b7af8290547467e8b94fd5d739bdf23"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "6b5252acdd5a6953e9abfbbbe32c522f"
server_key_iv: "7587f6a59cf46ef545cd90a86b483517"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.660203 18511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:42.677157 18511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:42.679913 18511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:42.680114 18511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:42.686692 18511 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:46597
I20260620 07:07:42.686936 18578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:46597 every 8 connection(s)
I20260620 07:07:42.687634 18511 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:07:42.690248 18579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:42.694669 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18511
I20260620 07:07:42.694765 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:07:42.695012 8288 external_mini_cluster.cc:1428] Setting key 41787886f7704379c381d191c9067805
I20260620 07:07:42.695470 18579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: Bootstrap starting.
I20260620 07:07:42.697746 18579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:42.698412 18579 log.cc:824] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:42.700465 18579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: No bootstrap required, opened a new log
Jun 20 07:07:42 dist-test-slave-wjw3 krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:42.704305 18579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } }
I20260620 07:07:42.704591 18579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:42.704648 18579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7af8290547467e8b94fd5d739bdf23, State: Initialized, Role: FOLLOWER
I20260620 07:07:42.705120 18579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } }
I20260620 07:07:42.705272 18579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:07:42.705343 18579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:07:42.705437 18579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:42.706478 18579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } }
I20260620 07:07:42.706776 18579 leader_election.cc:302] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b7af8290547467e8b94fd5d739bdf23; no voters:
I20260620 07:07:42.707056 18579 leader_election.cc:288] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:07:42.707134 18584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:42.707288 18584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [term 1 LEADER]: Becoming Leader. State: Replica: 4b7af8290547467e8b94fd5d739bdf23, State: Running, Role: LEADER
I20260620 07:07:42.707590 18584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } }
I20260620 07:07:42.708300 18579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:07:42.708714 18586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b7af8290547467e8b94fd5d739bdf23. Latest consensus state: current_term: 1 leader_uuid: "4b7af8290547467e8b94fd5d739bdf23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } } }
I20260620 07:07:42.708827 18586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:42.709228 18589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:07:42.709664 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:42.696312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.8.24.62:46597)
0620 07:07:42.696742 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:42.696756 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:42.696792 (+ 36us) server_negotiation.cc:407] Connection header received
0620 07:07:42.697399 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:42.697415 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:42.697731 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:42.698057 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:42.698826 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.700198 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.701062 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.701710 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.701878 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:42.704263 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:42.704270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:42.704286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:42.704317 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:42.706278 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:42.706674 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:42.706679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:42.706685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:42.706803 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:42.707063 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:42.707066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:42.707068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:42.707617 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:42.707818 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:42.708419 (+ 601us) server_negotiation.cc:299] Negotiation successful
0620 07:07:42.708704 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":140,"threads_started":1}
I20260620 07:07:42.711016 18585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b7af8290547467e8b94fd5d739bdf23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b7af8290547467e8b94fd5d739bdf23" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46597 } } }
I20260620 07:07:42.711344 18585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23 [sys.catalog]: This master's current role is: LEADER
I20260620 07:07:42.712262 18589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:07:42.717453 18589 catalog_manager.cc:1347] Generated new cluster ID: 71497f43f61e4b21ba3e2ace0c112e02
I20260620 07:07:42.717538 18589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:07:42.730667 18589 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:07:42.731583 18589 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:07:42.739980 18589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: Generated new TSK 0
I20260620 07:07:42.740679 18589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:07:42.767555 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:42.881209 18605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:42.881431 18605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:42.881474 18605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:42.885385 18605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:42.885432 18605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:42.885499 18605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:07:42.890532 18605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:42.891500 18605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:42.892442 18605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:42.901268 18611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:42.901275 18610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:42.901333 18605 server_base.cc:1034] running on GCE node
W20260620 07:07:42.901286 18613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:42.901796 18605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:42.902344 18605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:42.903487 18605 hybrid_clock.cc:648] HybridClock initialized: now 1781939262903471 us; error 27 us; skew 500 ppm
Jun 20 07:07:42 dist-test-slave-wjw3 krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939262, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:42.905905 18605 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:07:42.906666 18605 webserver.cc:460] Webserver started at http://127.8.24.1:36379/ using document root <none> and password file <none>
I20260620 07:07:42.907171 18605 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:42.907231 18605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:42.907413 18605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:42.909373 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "3a67e8ef216e4c4c8101ec43ce4b710a"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "43f7c2eeb8c5cbdcf84596ff729311fe"
server_key_iv: "797d067bd2999c24e2bf53c9751bd831"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.909898 18605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "3a67e8ef216e4c4c8101ec43ce4b710a"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "43f7c2eeb8c5cbdcf84596ff729311fe"
server_key_iv: "797d067bd2999c24e2bf53c9751bd831"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.913316 18605 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:07:42.915555 18620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:42.916662 18605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:07:42.916769 18605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "3a67e8ef216e4c4c8101ec43ce4b710a"
format_stamp: "Formatted at 2026-06-20 07:07:42 on dist-test-slave-wjw3"
server_key: "43f7c2eeb8c5cbdcf84596ff729311fe"
server_key_iv: "797d067bd2999c24e2bf53c9751bd831"
server_key_version: "encryptionkey@0"
I20260620 07:07:42.916854 18605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:42.928844 18605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:42.931406 18605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:42.931586 18605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:42.932127 18605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:42.932904 18605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:42.932956 18605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:42.933007 18605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:42.933036 18605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:42.941138 18605 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45007
I20260620 07:07:42.941162 18733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45007 every 8 connection(s)
I20260620 07:07:42.941974 18605 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:07:42.943531 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18605
I20260620 07:07:42.943636 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:07:42.943935 8288 external_mini_cluster.cc:1428] Setting key 69dde8c492efe1f6d26fbcd558b93bd4
Jun 20 07:07:42 dist-test-slave-wjw3 krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939262, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:07:42.955123 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:42.943646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57949 (local address 127.8.24.62:46597)
0620 07:07:42.943772 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:42.943776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:42.944812 (+ 1036us) server_negotiation.cc:407] Connection header received
0620 07:07:42.946238 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:42.946242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:42.946299 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:42.946382 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:42.947522 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.948052 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.949159 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.949388 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.949475 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:42.951955 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:42.951957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:42.951959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:42.951983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:42.953221 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:42.953644 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:42.953646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:42.953647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:42.953714 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:42.954030 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:42.954032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:42.954033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:42.954197 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:42.954282 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:42.954838 (+ 556us) server_negotiation.cc:299] Negotiation successful
0620 07:07:42.954963 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:42.955894 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:42.944147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46597 (local address 127.8.24.1:57949)
0620 07:07:42.944657 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:42.944710 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:42.946021 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:42.946536 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:42.946542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:42.946901 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:42.947366 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:42.947379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.948168 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.948172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:42.949042 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:42.949051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:42.949528 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:42.949532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:42.949685 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:42.950247 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:42.950263 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:42.951793 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:42.953324 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:42.953329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:42.953340 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:42.953571 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:42.953787 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:42.953789 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:42.953790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:42.953938 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:42.954296 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:42.954301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:42.954593 (+ 292us) client_negotiation.cc:769] Sending connection context
0620 07:07:42.954772 (+ 179us) client_negotiation.cc:240] Negotiation successful
0620 07:07:42.955035 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":126,"threads_started":1}
I20260620 07:07:42.957052 18734 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46597
I20260620 07:07:42.957266 18734 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:42.957713 18734 heartbeater.cc:511] Master 127.8.24.62:46597 requested a full tablet report, sending...
I20260620 07:07:42.959362 18543 ts_manager.cc:194] Registered new tserver with Master: 3a67e8ef216e4c4c8101ec43ce4b710a (127.8.24.1:45007)
I20260620 07:07:42.960956 18543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:57949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:07:42.967705 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:43.073040 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:43.073269 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:43.073309 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:43.077219 18739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:43.077272 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:43.077328 18739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:07:43.082331 18739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:43.083235 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:43.084209 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:43.095541 18744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:43.095611 18739 server_base.cc:1034] running on GCE node
W20260620 07:07:43.095737 18747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:43.097558 18745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:43.098031 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:43.098794 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:43.099975 18739 hybrid_clock.cc:648] HybridClock initialized: now 1781939263099905 us; error 83 us; skew 500 ppm
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939263, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:43.103291 18739 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:07:43.104316 18739 webserver.cc:460] Webserver started at http://127.8.24.2:42945/ using document root <none> and password file <none>
I20260620 07:07:43.104986 18739 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:43.105072 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:43.105329 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:43.107877 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "f72c420ccf3f419f80d6b4a13fc08847"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "5f8f85061308e4a997d5874f1b716b3d"
server_key_iv: "9f74eb2c245dc8d7803879e9a02a9abd"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.108450 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "f72c420ccf3f419f80d6b4a13fc08847"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "5f8f85061308e4a997d5874f1b716b3d"
server_key_iv: "9f74eb2c245dc8d7803879e9a02a9abd"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.112017 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:07:43.114267 18754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.115412 18739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:07:43.115562 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "f72c420ccf3f419f80d6b4a13fc08847"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "5f8f85061308e4a997d5874f1b716b3d"
server_key_iv: "9f74eb2c245dc8d7803879e9a02a9abd"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.115648 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:43.124794 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:43.127303 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:43.127507 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:43.128046 18739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:43.128813 18739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:43.128856 18739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.128907 18739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:43.128933 18739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.138052 18739 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38909
I20260620 07:07:43.138098 18867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38909 every 8 connection(s)
I20260620 07:07:43.138895 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:07:43.142992 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18739
I20260620 07:07:43.143093 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:07:43.143335 8288 external_mini_cluster.cc:1428] Setting key 75a5af2c3922ce83bdffad65315b4117
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939263, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:07:43.152297 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.140543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:38963 (local address 127.8.24.62:46597)
0620 07:07:43.140671 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.140675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.141473 (+ 798us) server_negotiation.cc:407] Connection header received
0620 07:07:43.142749 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.142753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.142808 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.142880 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:43.144161 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.144987 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.146199 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.146422 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.146505 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.148991 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:43.148995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:43.148996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:43.149020 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:43.150325 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.150870 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.150872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.150873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.150939 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.151261 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.151263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.151264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.151416 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:43.151519 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.152061 (+ 542us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.152162 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:43.153126 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.140902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46597 (local address 127.8.24.2:38963)
0620 07:07:43.141333 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.141378 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.142556 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.143057 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.143063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.143471 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:43.144002 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.144022 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.145132 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.145136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.146087 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.146097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.146547 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.146551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.146715 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.147234 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:43.147250 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:43.148835 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:43.150436 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:43.150442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:43.150457 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:43.150784 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:43.151013 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:43.151016 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:43.151018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:43.151180 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:43.151521 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:43.151528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:43.151831 (+ 303us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.152036 (+ 205us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.152358 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260620 07:07:43.154129 18868 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:07:43.154359 18868 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:43.154808 18868 heartbeater.cc:511] Master 127.8.24.62:46597 requested a full tablet report, sending...
I20260620 07:07:43.155839 18543 ts_manager.cc:194] Registered new tserver with Master: f72c420ccf3f419f80d6b4a13fc08847 (127.8.24.2:38909)
I20260620 07:07:43.156373 18543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:38963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:07:43.167299 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:07:43.274399 18873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:07:43.274677 18873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:07:43.274734 18873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:07:43.278986 18873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:07:43.279040 18873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:07:43.279098 18873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:07:43.284894 18873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:46597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:07:43.286134 18873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:07:43.287470 18873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:07:43.297950 18878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:43.297988 18881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:07:43.298372 18879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:07:43.298350 18873 server_base.cc:1034] running on GCE node
I20260620 07:07:43.298786 18873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:07:43.299546 18873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:07:43.300722 18873 hybrid_clock.cc:648] HybridClock initialized: now 1781939263300691 us; error 38 us; skew 500 ppm
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939263, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:07:43.303560 18873 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:07:43.304364 18873 webserver.cc:460] Webserver started at http://127.8.24.3:41987/ using document root <none> and password file <none>
I20260620 07:07:43.304893 18873 fs_manager.cc:359] Metadata directory not provided
I20260620 07:07:43.304960 18873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:07:43.305145 18873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:07:43.307049 18873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "8f27c8d8743746f382b502140e99a351"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "69cca3a106055e96307296be15f61ed7"
server_key_iv: "a91af4763f0eff7c5e54d293592e37b2"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.307615 18873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "8f27c8d8743746f382b502140e99a351"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "69cca3a106055e96307296be15f61ed7"
server_key_iv: "a91af4763f0eff7c5e54d293592e37b2"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.311218 18873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:07:43.313398 18888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.314544 18873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 07:07:43.314643 18873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "8f27c8d8743746f382b502140e99a351"
format_stamp: "Formatted at 2026-06-20 07:07:43 on dist-test-slave-wjw3"
server_key: "69cca3a106055e96307296be15f61ed7"
server_key_iv: "a91af4763f0eff7c5e54d293592e37b2"
server_key_version: "encryptionkey@0"
I20260620 07:07:43.314725 18873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:07:43.340565 18873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:07:43.343297 18873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:07:43.343500 18873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:07:43.344085 18873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:07:43.344875 18873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:07:43.344920 18873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.344985 18873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:07:43.345009 18873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:07:43.353113 18873 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:37917
I20260620 07:07:43.353145 19001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:37917 every 8 connection(s)
I20260620 07:07:43.353942 18873 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:07:43.362689 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 18873
I20260620 07:07:43.362778 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939263, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:07:43.363020 8288 external_mini_cluster.cc:1428] Setting key 43e6898b2c2f74bc1a58bc943fdc34fd
I20260620 07:07:43.366703 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.355687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54717 (local address 127.8.24.62:46597)
0620 07:07:43.355811 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.355815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.356603 (+ 788us) server_negotiation.cc:407] Connection header received
0620 07:07:43.357908 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.357912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.357971 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.358044 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:43.359183 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.359777 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.360873 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.361093 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.361183 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.363525 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:43.363528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:43.363530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:43.363555 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:43.364917 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.365353 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.365356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.365358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.365438 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.365723 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.365726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.365728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.365900 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:43.365980 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.366428 (+ 448us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.366548 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:07:43.367424 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.356039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:46597 (local address 127.8.24.3:54717)
0620 07:07:43.356465 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.356506 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.357701 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.358226 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.358233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.358569 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:07:43.359039 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.359053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.359893 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.359896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.360753 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.360761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.361226 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.361229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.361384 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.361879 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:07:43.361894 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:07:43.363347 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:07:43.365018 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:43.365024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:43.365038 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:43.365265 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:43.365514 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:07:43.365517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:07:43.365519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:07:43.365644 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:07:43.365990 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:07:43.365997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:07:43.366217 (+ 220us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.366372 (+ 155us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.366662 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":108,"threads_started":1}
I20260620 07:07:43.368433 19002 heartbeater.cc:347] Connected to a master server at 127.8.24.62:46597
I20260620 07:07:43.368633 19002 heartbeater.cc:464] Registering TS with master...
I20260620 07:07:43.369050 19002 heartbeater.cc:511] Master 127.8.24.62:46597 requested a full tablet report, sending...
I20260620 07:07:43.369948 18543 ts_manager.cc:194] Registered new tserver with Master: 8f27c8d8743746f382b502140e99a351 (127.8.24.3:37917)
I20260620 07:07:43.370471 18543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:54717
I20260620 07:07:43.376493 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:07:43.384420 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.378193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.8.24.62:46597)
0620 07:07:43.378308 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.378311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.378477 (+ 166us) server_negotiation.cc:407] Connection header received
0620 07:07:43.378580 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.378582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.378621 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.378675 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:07:43.379250 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.379799 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.380565 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.380759 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.380822 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.382022 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:07:43.382025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:07:43.382027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:07:43.382048 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:07:43.383134 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.383514 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.383517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.383519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.383588 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:07:43.383783 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:07:43.383785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:07:43.383786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:07:43.383985 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:07:43.384081 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.384224 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.384307 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:07:43.386906 18543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:07:43.389832 18543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:07:43.402639 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.397613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45007 (local address 127.0.0.1:44240)
0620 07:07:43.398239 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.398252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.398347 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.398960 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.398965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.398989 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:43.399176 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.399184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.400328 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.400331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.401340 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.401349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.402144 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.402147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.402247 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.402280 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.402387 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.402501 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":36,"spinlock_wait_cycles":2432,"thread_start_us":89,"threads_started":1}
I20260620 07:07:43.402647 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.398141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.8.24.1:45007)
0620 07:07:43.398493 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.398504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.398524 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:07:43.398585 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.398589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.398793 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.398955 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:43.399298 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.400218 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.401462 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.402036 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.402132 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.402276 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.402417 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.402506 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":85,"threads_started":1}
I20260620 07:07:43.403944 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.397342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38909 (local address 127.0.0.1:59032)
0620 07:07:43.397711 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.397761 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.397868 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.399009 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.399012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.399044 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:43.399189 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.399195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.401146 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.401150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.402471 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.402482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.403546 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.403550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.403638 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.403660 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.403743 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.403807 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":48,"spinlock_wait_cycles":768,"thread_start_us":91,"threads_started":1}
I20260620 07:07:43.404134 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.398056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37917 (local address 127.0.0.1:49178)
0620 07:07:43.399195 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.399207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.399283 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.399703 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.399706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.399726 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:43.399796 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.399801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.401161 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.401163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.402471 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.402482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.403771 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.403774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.403855 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.403872 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.403935 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.404005 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":128,"thread_start_us":75,"threads_started":1}
I20260620 07:07:43.404191 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.398196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.8.24.3:37917)
0620 07:07:43.398792 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.398802 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.399352 (+ 550us) server_negotiation.cc:407] Connection header received
0620 07:07:43.399409 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.399412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.399587 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.399689 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:43.399918 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.401040 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.402603 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.403584 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.403670 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.403778 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.403979 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.404066 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":120,"threads_started":1}
I20260620 07:07:43.404325 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.397494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.8.24.2:38909)
0620 07:07:43.398573 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.398584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.398606 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:07:43.398681 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.398686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.398881 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.399015 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:43.399823 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.400997 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.402617 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.403381 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.403547 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.403688 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.404094 (+ 406us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.404193 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":96,"threads_started":1}
I20260620 07:07:43.405723 18668 tablet_service.cc:1467] Processing CreateTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb (DEFAULT_TABLE table=test-table [id=fb72b9d1f60c4064b9b3435bfa5091e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:43.406651 18668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:43.407179 18936 tablet_service.cc:1467] Processing CreateTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb (DEFAULT_TABLE table=test-table [id=fb72b9d1f60c4064b9b3435bfa5091e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:43.407178 18802 tablet_service.cc:1467] Processing CreateTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb (DEFAULT_TABLE table=test-table [id=fb72b9d1f60c4064b9b3435bfa5091e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:07:43.408212 18802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:43.408219 18936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:07:43.412546 19019 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Bootstrap starting.
I20260620 07:07:43.414834 19019 tablet_bootstrap.cc:654] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:43.414986 19020 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Bootstrap starting.
I20260620 07:07:43.415952 19019 log.cc:824] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:43.416121 19021 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Bootstrap starting.
I20260620 07:07:43.417157 19020 tablet_bootstrap.cc:654] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:43.418167 19019 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: No bootstrap required, opened a new log
I20260620 07:07:43.418263 19020 log.cc:824] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:43.418370 19019 ts_tablet_manager.cc:1397] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 07:07:43.418551 19021 tablet_bootstrap.cc:654] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Neither blocks nor log segments found. Creating new log.
I20260620 07:07:43.419397 19021 log.cc:824] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Log is configured to *not* fsync() on all Append() calls
I20260620 07:07:43.420163 19020 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: No bootstrap required, opened a new log
I20260620 07:07:43.420342 19020 ts_tablet_manager.cc:1397] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:07:43.421432 19021 tablet_bootstrap.cc:492] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: No bootstrap required, opened a new log
I20260620 07:07:43.421605 19021 ts_tablet_manager.cc:1397] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:07:43.422375 19019 raft_consensus.cc:348] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.422652 19019 raft_consensus.cc:374] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:43.422706 19019 raft_consensus.cc:729] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a67e8ef216e4c4c8101ec43ce4b710a, State: Initialized, Role: FOLLOWER
I20260620 07:07:43.423143 19019 consensus_queue.cc:260] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.423398 19020 raft_consensus.cc:348] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.423676 19020 raft_consensus.cc:374] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:43.423732 19020 raft_consensus.cc:729] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f27c8d8743746f382b502140e99a351, State: Initialized, Role: FOLLOWER
I20260620 07:07:43.423990 18734 heartbeater.cc:503] Master 127.8.24.62:46597 was elected leader, sending a full tablet report...
I20260620 07:07:43.424189 19019 ts_tablet_manager.cc:1428] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260620 07:07:43.424185 19020 consensus_queue.cc:260] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.424804 19002 heartbeater.cc:503] Master 127.8.24.62:46597 was elected leader, sending a full tablet report...
I20260620 07:07:43.425073 19020 ts_tablet_manager.cc:1428] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:07:43.425159 19021 raft_consensus.cc:348] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.425379 19021 raft_consensus.cc:374] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:07:43.425423 19021 raft_consensus.cc:729] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72c420ccf3f419f80d6b4a13fc08847, State: Initialized, Role: FOLLOWER
I20260620 07:07:43.425845 19021 consensus_queue.cc:260] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.426469 18868 heartbeater.cc:503] Master 127.8.24.62:46597 was elected leader, sending a full tablet report...
I20260620 07:07:43.426745 19021 ts_tablet_manager.cc:1428] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 07:07:43.443007 18735 tablet.cc:2369] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:07:43.535120 19027 raft_consensus.cc:482] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:07:43.535323 19027 raft_consensus.cc:504] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.536499 19027 leader_election.cc:288] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f27c8d8743746f382b502140e99a351 (127.8.24.3:37917), 3a67e8ef216e4c4c8101ec43ce4b710a (127.8.24.1:45007)
I20260620 07:07:43.541090 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.536963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40347 (local address 127.8.24.1:45007)
0620 07:07:43.537080 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.537084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.537226 (+ 142us) server_negotiation.cc:407] Connection header received
0620 07:07:43.537346 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.537350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.537388 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.537443 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:43.537663 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.538382 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.539917 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.540518 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.540622 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.540683 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.540894 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.540961 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:07:43.541090 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.536838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37345 (local address 127.8.24.3:37917)
0620 07:07:43.536959 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.536963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.536978 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:07:43.537125 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.537129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.537174 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.537243 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:07:43.537508 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.538269 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.539917 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.540518 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.540622 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.540682 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.540894 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.540962 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:07:43.541108 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.536741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:37917 (local address 127.8.24.2:37345)
0620 07:07:43.536887 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.536903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.536993 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.537282 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.537286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.537300 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:43.537405 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.537410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.538378 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.538380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.539801 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.539811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.540625 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.540628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.540760 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.540799 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.540873 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.540959 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":129}
I20260620 07:07:43.541115 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.536757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45007 (local address 127.8.24.2:40347)
0620 07:07:43.537140 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.537150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:07:43.537243 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:07:43.537476 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:07:43.537478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:07:43.537495 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:07:43.537580 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.537584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.538471 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.538474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.539804 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:07:43.539811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.540625 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.540628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:07:43.540794 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.540812 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:07:43.540873 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:07:43.540959 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":110,"thread_start_us":136,"threads_started":1}
I20260620 07:07:43.541857 18688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6889f1e6a6c6417a9125c52c0c5b2bbb" candidate_uuid: "f72c420ccf3f419f80d6b4a13fc08847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" is_pre_election: true
I20260620 07:07:43.541812 18956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6889f1e6a6c6417a9125c52c0c5b2bbb" candidate_uuid: "f72c420ccf3f419f80d6b4a13fc08847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f27c8d8743746f382b502140e99a351" is_pre_election: true
I20260620 07:07:43.542147 18688 raft_consensus.cc:2454] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f72c420ccf3f419f80d6b4a13fc08847 in term 0.
I20260620 07:07:43.542147 18956 raft_consensus.cc:2454] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f72c420ccf3f419f80d6b4a13fc08847 in term 0.
I20260620 07:07:43.542608 18755 leader_election.cc:302] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f27c8d8743746f382b502140e99a351, f72c420ccf3f419f80d6b4a13fc08847; no voters:
I20260620 07:07:43.542829 19027 raft_consensus.cc:2781] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:07:43.542906 19027 raft_consensus.cc:482] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:07:43.542951 19027 raft_consensus.cc:3037] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:43.543972 19027 raft_consensus.cc:504] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.544288 19027 leader_election.cc:288] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [CANDIDATE]: Term 1 election: Requested vote from peers 8f27c8d8743746f382b502140e99a351 (127.8.24.3:37917), 3a67e8ef216e4c4c8101ec43ce4b710a (127.8.24.1:45007)
I20260620 07:07:43.544660 18688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6889f1e6a6c6417a9125c52c0c5b2bbb" candidate_uuid: "f72c420ccf3f419f80d6b4a13fc08847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a"
I20260620 07:07:43.544652 18956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6889f1e6a6c6417a9125c52c0c5b2bbb" candidate_uuid: "f72c420ccf3f419f80d6b4a13fc08847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f27c8d8743746f382b502140e99a351"
I20260620 07:07:43.544776 18688 raft_consensus.cc:3037] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:43.544776 18956 raft_consensus.cc:3037] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:07:43.545965 18688 raft_consensus.cc:2454] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f72c420ccf3f419f80d6b4a13fc08847 in term 1.
I20260620 07:07:43.545962 18956 raft_consensus.cc:2454] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f72c420ccf3f419f80d6b4a13fc08847 in term 1.
I20260620 07:07:43.546236 18755 leader_election.cc:302] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a67e8ef216e4c4c8101ec43ce4b710a, 8f27c8d8743746f382b502140e99a351, f72c420ccf3f419f80d6b4a13fc08847; no voters:
I20260620 07:07:43.546430 19027 raft_consensus.cc:2781] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:07:43.546641 19027 raft_consensus.cc:686] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 1 LEADER]: Becoming Leader. State: Replica: f72c420ccf3f419f80d6b4a13fc08847, State: Running, Role: LEADER
I20260620 07:07:43.546949 19027 consensus_queue.cc:237] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } }
I20260620 07:07:43.550117 18543 catalog_manager.cc:5495] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 reported cstate change: term changed from 0 to 1, leader changed from <none> to f72c420ccf3f419f80d6b4a13fc08847 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "f72c420ccf3f419f80d6b4a13fc08847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f72c420ccf3f419f80d6b4a13fc08847" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 } health_report { overall_health: UNKNOWN } } }
I20260620 07:07:43.560030 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.557027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.8.24.2:38909)
0620 07:07:43.557126 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:07:43.557129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:07:43.557140 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:07:43.557257 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:07:43.557259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:07:43.557309 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:07:43.557393 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:07:43.557574 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.558061 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.558798 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:07:43.558999 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:07:43.559062 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:07:43.559270 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:07:43.559709 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:07:43.559792 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:07:43.559856 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 07:07:43.559903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:07:43.565411 18688 raft_consensus.cc:1261] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 3a67e8ef216e4c4c8101ec43ce4b710a [term 1 FOLLOWER]: Refusing update from remote peer f72c420ccf3f419f80d6b4a13fc08847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:43.565434 18956 raft_consensus.cc:1261] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 1 FOLLOWER]: Refusing update from remote peer f72c420ccf3f419f80d6b4a13fc08847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:07:43.566078 19029 consensus_queue.cc:1035] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a67e8ef216e4c4c8101ec43ce4b710a" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:43.566241 19027 consensus_queue.cc:1035] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f27c8d8743746f382b502140e99a351" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:07:43.572288 19032 mvcc.cc:205] Tried to move back new op lower bound from 7298823223557234688 to 7298823223490863104. Current Snapshot: MvccSnapshot[applied={T|T < 7298823223557234688}]
I20260620 07:07:43.574962 19033 mvcc.cc:205] Tried to move back new op lower bound from 7298823223557234688 to 7298823223490863104. Current Snapshot: MvccSnapshot[applied={T|T < 7298823223557234688}]
I20260620 07:07:43.581342 18542 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:43.581483 18542 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:07:43.583694 18542 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b7af8290547467e8b94fd5d739bdf23: Sending DeleteTablet for 3 replicas of tablet 6889f1e6a6c6417a9125c52c0c5b2bbb
I20260620 07:07:43.584404 18668 tablet_service.cc:1514] Processing DeleteTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:44240
I20260620 07:07:43.584475 18936 tablet_service.cc:1514] Processing DeleteTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49178
I20260620 07:07:43.584558 18802 tablet_service.cc:1514] Processing DeleteTablet for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:07:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:59032
I20260620 07:07:43.585239 19043 tablet_replica.cc:330] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: stopping tablet replica
I20260620 07:07:43.585273 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18605
I20260620 07:07:43.585520 19043 raft_consensus.cc:2229] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:07:43.585916 19043 raft_consensus.cc:2258] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:43.586232 19042 tablet_replica.cc:330] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: stopping tablet replica
I20260620 07:07:43.586516 19042 raft_consensus.cc:2229] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:07:43.586789 19042 raft_consensus.cc:2258] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:07:43.588019 19042 ts_tablet_manager.cc:1905] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:43.588523 19043 ts_tablet_manager.cc:1905] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:07:43.590493 19043 ts_tablet_manager.cc:1918] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:43.590512 19042 ts_tablet_manager.cc:1918] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:07:43.590575 19043 log.cc:1196] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/6889f1e6a6c6417a9125c52c0c5b2bbb
I20260620 07:07:43.590586 19042 log.cc:1196] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/6889f1e6a6c6417a9125c52c0c5b2bbb
I20260620 07:07:43.590862 19043 ts_tablet_manager.cc:1939] T 6889f1e6a6c6417a9125c52c0c5b2bbb P f72c420ccf3f419f80d6b4a13fc08847: Deleting consensus metadata
I20260620 07:07:43.590875 19042 ts_tablet_manager.cc:1939] T 6889f1e6a6c6417a9125c52c0c5b2bbb P 8f27c8d8743746f382b502140e99a351: Deleting consensus metadata
W20260620 07:07:43.591738 18529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:45007 (error 108)
I20260620 07:07:43.591959 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18739
I20260620 07:07:43.593232 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.592964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45007 (local address 127.0.0.1:44256)
0620 07:07:43.593085 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.593152 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 07:07:43.593649 18529 catalog_manager.cc:4584] TS 3a67e8ef216e4c4c8101ec43ce4b710a (127.8.24.1:45007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6889f1e6a6c6417a9125c52c0c5b2bbb: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45007: connect: Connection refused (error 111)
I20260620 07:07:43.594121 18527 catalog_manager.cc:4841] TS 8f27c8d8743746f382b502140e99a351 (127.8.24.3:37917): tablet 6889f1e6a6c6417a9125c52c0c5b2bbb (table test-table [id=fb72b9d1f60c4064b9b3435bfa5091e5]) successfully deleted
I20260620 07:07:43.596952 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18873
I20260620 07:07:43.600123 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:07:43.597060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38909 (local address 127.0.0.1:59050)
0620 07:07:43.599999 (+ 2939us) negotiation.cc:107] Waiting for socket to connect
0620 07:07:43.600053 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2861}
I20260620 07:07:43.601513 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 18511
2026-06-20T07:07:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2136 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[19047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[19047](info): set up 2 sockets
Jun 20 07:07:43 dist-test-slave-wjw3 krb5kdc[19047](info): commencing operation
krb5kdc: starting...
W20260620 07:07:44.548125 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:07:44 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:07:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
I20260620 07:07:44.689536 8288 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksJiSS1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/postgres -l logfile start
2026-06-20 07:07:46.758 UTC [19077] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 07:07:46.758 UTC [19077] LOG: listening on IPv4 address "127.8.24.20", port 38725
2026-06-20 07:07:46.761 UTC [19077] LOG: listening on Unix socket "/tmp/.s.PGSQL.38725"
2026-06-20 07:07:46.787 UTC [19080] LOG: database system was shut down at 2026-06-20 07:07:45 UTC
2026-06-20 07:07:46.791 UTC [19077] LOG: database system is ready to accept connections
I20260620 07:07:47.663865 8288 mini_postgres.cc:96] Postgres bound to 38725
2026-06-20 07:07:47.669 UTC [19088] FATAL: database "slave" does not exist
127.8.24.20:38725 - accepting connections
I20260620 07:07:47.670035 8288 mini_ranger.cc:161] Starting Ranger...
I20260620 07:07:47.683523 8288 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 07:07:48.057410 8288 mini_ranger.cc:87] Created ranger Postgres database
I20260620 07:07:48.057494 8288 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin
2026-06-20 07:07:48,320 [I] DB FLAVOR :POSTGRES
2026-06-20 07:07:48,320 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 07:07:48,321 [I] Checking connection..
2026-06-20 07:07:48,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 07:07:48,549 [I] Checking connection passed.
2026-06-20 07:07:48,549 [I] --------- Verifying version history table ---------
2026-06-20 07:07:48,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:07:48,776 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 07:07:48,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:07:49,026 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 07:07:49,027 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 07:07:49,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 07:07:49.264 UTC [19152] WARNING: there is no transaction in progress
2026-06-20 07:07:49,269 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 07:07:49,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:07:49,492 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 07:07:49,492 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 07:07:49,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 07:07:49,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 07:07:49,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wjw3.c.gcp-upstream.internal','N') ;"
2026-06-20 07:07:50,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 07:07:50,453 [I] Table x_portal_user does not exist in database ranger
2026-06-20 07:07:50,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 07:07:50,703 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 07:07:50,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 07:07:50,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 07:07:51,197 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 07:07:51,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 07:07:52.066 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.080 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.094 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.108 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.121 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.140 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.189 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.195 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.208 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.217 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.226 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.238 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.246 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.252 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.258 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52.263 UTC [19260] WARNING: there is no transaction in progress
2026-06-20 07:07:52,783 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 07:07:52,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 07:07:53,020 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 07:07:53,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 07:07:53,264 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 07:07:53,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 07:07:53,515 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 07:07:53,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 07:07:53,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 07:07:53,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wjw3.c.gcp-upstream.internal';"
2026-06-20 07:07:54,233 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 07:07:54,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 07:07:54,460 [I] DB_PATCHES have already been applied
I20260620 07:07:54.465700 8288 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksJiSS1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 07:07:54.465767 8288 mini_ranger.cc:193] Using host: 127.8.24.20
I20260620 07:07:54.468998 8288 mini_ranger.cc:227] Ranger admin URL: http://127.8.24.20:34491
Jun 20, 2026 7:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 7:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 7:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 7:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 7:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 7:07:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 7:07:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 07:07:55.470849 8288 mini_ranger.cc:160] Time spent starting Ranger: real 7.801s user 0.005s sys 0.004s
Jun 20, 2026 7:08:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 7:08:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 7:08:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 7:08:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 7:08:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 07:08:16.719537 8288 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.24.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 07:08:16 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939296, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 07:08:16.794175 8288 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 07:08:16.802683 8288 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 07:08:17.465193 8288 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 07:08:17.465286 8288 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms
I20260620 07:08:17.465379 8288 mini_ranger_kms.cc:227] Using postgres at 127.8.24.20:38725
2026-06-20 07:08:17,665 [I] DB FLAVOR :POSTGRES
2026-06-20 07:08:17,665 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 07:08:17,665 [I] Checking connection
2026-06-20 07:08:17,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 07:08:17,921 [I] connection success
2026-06-20 07:08:17,921 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 07:08:17,921 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 07:08:17,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 07:08:18,217 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 07:08:18,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 07:08:18,218 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 07:08:18,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:38725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 07:08:18,476 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 07:08:18.904341 8288 mini_ranger_kms.cc:310] Created kms service
I20260620 07:08:19.108191 8288 mini_ranger_kms.cc:326] Created kudu user
I20260620 07:08:19.150456 8288 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 07:08:19 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939296, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:19.523566 8288 mini_ranger_kms.cc:384] Added ranger policy
I20260620 07:08:19.523789 8288 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 07:08:19.523811 8288 mini_ranger_kms.cc:240] Using host: 127.8.24.20
I20260620 07:08:19.527396 8288 mini_ranger_kms.cc:276] Ranger KMS PID: 19440
I20260620 07:08:19.527556 8288 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.24.20:50713
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 7:08:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 07:08:21.579252 8288 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.785s user 0.019s sys 0.166s
I20260620 07:08:21.579388 8288 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 07:08:21.579526 8288 mini_ranger_kms.cc:401] 127.8.24.20:50713/kms/v1/keys
Jun 20 07:08:21 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781939301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 07:08:22 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Jun 20 07:08:22 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Jun 20 07:08:24 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:08:24.726163 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40879
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:40879
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:50713/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:08:24.835840 19491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:08:24.836117 19491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:08:24.836153 19491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:08:24.840145 19491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:08:24.840205 19491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:08:24.840224 19491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:08:24.840237 19491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:08:24.840251 19491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:08:24.845734 19491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.24.20:50713/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:40879
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:40879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:08:24.846815 19491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:08:24.847846 19491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:08:24.856321 19497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:24.856405 19496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:24.856475 19499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:24.856525 19491 server_base.cc:1034] running on GCE node
I20260620 07:08:24.857298 19491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:08:24.858686 19491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:08:24.859859 19491 hybrid_clock.cc:648] HybridClock initialized: now 1781939304859823 us; error 45 us; skew 500 ppm
Jun 20 07:08:24 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939304, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:08:24.867678 19491 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:08:24.868862 19491 webserver.cc:460] Webserver started at http://127.8.24.62:38235/ using document root <none> and password file <none>
I20260620 07:08:24.869537 19491 fs_manager.cc:359] Metadata directory not provided
I20260620 07:08:24.869611 19491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:08:24.869836 19491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:08:24 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939304, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:24.924242 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "29562d197010470c9df4bc84ed91d271"
format_stamp: "Formatted at 2026-06-20 07:08:24 on dist-test-slave-wjw3"
server_key: "51ba9d69ebb0d05b77d2e4131f8821b4"
server_key_iv: "801022c24e0bf59450d7125b06a02e00"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:24.924952 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "29562d197010470c9df4bc84ed91d271"
format_stamp: "Formatted at 2026-06-20 07:08:24 on dist-test-slave-wjw3"
server_key: "51ba9d69ebb0d05b77d2e4131f8821b4"
server_key_iv: "801022c24e0bf59450d7125b06a02e00"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:24.929572 19491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 07:08:24.946825 19506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:24.948686 19491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 07:08:24.948836 19491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "29562d197010470c9df4bc84ed91d271"
format_stamp: "Formatted at 2026-06-20 07:08:24 on dist-test-slave-wjw3"
server_key: "51ba9d69ebb0d05b77d2e4131f8821b4"
server_key_iv: "801022c24e0bf59450d7125b06a02e00"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:24.948976 19491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:08:24.980010 19491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:08:24.982815 19491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:08:24.982987 19491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:08:24.990203 19491 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:40879
I20260620 07:08:24.990242 19558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:40879 every 8 connection(s)
I20260620 07:08:24.991171 19491 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:08:24.992739 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 19491
I20260620 07:08:24.992851 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:08:24.993109 8288 external_mini_cluster.cc:1428] Setting key 7b90b743c19afa715df8ce3935a20b9e
I20260620 07:08:24.994517 19559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:08:25.001219 19559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: Bootstrap starting.
Jun 20 07:08:24 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:08:25.003767 19559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: Neither blocks nor log segments found. Creating new log.
I20260620 07:08:25.005013 19559 log.cc:824] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: Log is configured to *not* fsync() on all Append() calls
I20260620 07:08:25.006392 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:24.994392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.8.24.62:40879)
0620 07:08:24.994851 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:24.994865 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:24.995158 (+ 293us) server_negotiation.cc:407] Connection header received
0620 07:08:24.995925 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:24.995942 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:24.996318 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:24.996625 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:08:24.997377 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:24.998343 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:24.999167 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:24.999671 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:24.999811 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:25.002094 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:08:25.002098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:08:25.002111 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:08:25.002136 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:08:25.003606 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:25.004037 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:25.004042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:25.004046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:25.004150 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:25.004401 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:25.004405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:25.004406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:25.004779 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:08:25.004936 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:25.005273 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 07:08:25.005536 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":112,"threads_started":1}
I20260620 07:08:25.007139 19559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: No bootstrap required, opened a new log
I20260620 07:08:25.010430 19559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } }
I20260620 07:08:25.010649 19559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:08:25.010684 19559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29562d197010470c9df4bc84ed91d271, State: Initialized, Role: FOLLOWER
I20260620 07:08:25.011087 19559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } }
I20260620 07:08:25.011202 19559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:08:25.011260 19559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:08:25.011348 19559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:08:25.012244 19559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } }
I20260620 07:08:25.012552 19559 leader_election.cc:302] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29562d197010470c9df4bc84ed91d271; no voters:
I20260620 07:08:25.012821 19559 leader_election.cc:288] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:08:25.012930 19564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:08:25.013116 19564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [term 1 LEADER]: Becoming Leader. State: Replica: 29562d197010470c9df4bc84ed91d271, State: Running, Role: LEADER
I20260620 07:08:25.013379 19564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } }
I20260620 07:08:25.014083 19559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:08:25.014518 19565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29562d197010470c9df4bc84ed91d271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } } }
I20260620 07:08:25.014691 19565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [sys.catalog]: This master's current role is: LEADER
I20260620 07:08:25.014937 19565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29562d197010470c9df4bc84ed91d271. Latest consensus state: current_term: 1 leader_uuid: "29562d197010470c9df4bc84ed91d271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29562d197010470c9df4bc84ed91d271" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 40879 } } }
I20260620 07:08:25.015058 19565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271 [sys.catalog]: This master's current role is: LEADER
I20260620 07:08:25.015329 19570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:08:25.018334 19570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:08:25.018818 19559 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 07:08:25.033408 19570 catalog_manager.cc:1347] Generated new cluster ID: 4e46628b3d6d4c0f80df6c14e9e3a8e5
I20260620 07:08:25.033491 19570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:08:25.049149 19570 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:08:25.050653 19570 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:08:25.061656 19570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: Generated new TSK 0
I20260620 07:08:25.062638 19570 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:08:25.778759 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:50713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:08:25.892234 19604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:08:25.892500 19604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:08:25.892542 19604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:08:25.896744 19604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:08:25.896812 19604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:08:25.896907 19604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:08:25.902276 19604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.24.20:50713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:08:25.903280 19604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:08:25.904309 19604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:08:25.914376 19609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:25.914373 19610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:25.914480 19604 server_base.cc:1034] running on GCE node
W20260620 07:08:25.914688 19612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:25.915263 19604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:08:25.916062 19604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:08:25.917253 19604 hybrid_clock.cc:648] HybridClock initialized: now 1781939305917229 us; error 39 us; skew 500 ppm
Jun 20 07:08:25 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939305, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:08:25.920780 19604 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:08:25.921828 19604 webserver.cc:460] Webserver started at http://127.8.24.1:42015/ using document root <none> and password file <none>
I20260620 07:08:25.922540 19604 fs_manager.cc:359] Metadata directory not provided
I20260620 07:08:25.922677 19604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:08:25.922919 19604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:08:25 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939305, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:25.942440 19604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "c40b360ab5f4428ea97bbf40fac26f77"
format_stamp: "Formatted at 2026-06-20 07:08:25 on dist-test-slave-wjw3"
server_key: "843dc2c84be097f125257137cc522c34"
server_key_iv: "a0a262711accab6c6c5310fb2ec6f408"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:25.944440 19604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "c40b360ab5f4428ea97bbf40fac26f77"
format_stamp: "Formatted at 2026-06-20 07:08:25 on dist-test-slave-wjw3"
server_key: "843dc2c84be097f125257137cc522c34"
server_key_iv: "a0a262711accab6c6c5310fb2ec6f408"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:25.948801 19604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 07:08:25.968941 19621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:25.970738 19604 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260620 07:08:25.970880 19604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "c40b360ab5f4428ea97bbf40fac26f77"
format_stamp: "Formatted at 2026-06-20 07:08:25 on dist-test-slave-wjw3"
server_key: "843dc2c84be097f125257137cc522c34"
server_key_iv: "a0a262711accab6c6c5310fb2ec6f408"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:25.971022 19604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:08:25.993909 19604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:08:25.997503 19604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:08:25.997726 19604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:08:25.998437 19604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:08:25.999524 19604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:08:25.999631 19604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:25.999758 19604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:08:25.999845 19604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.014578 19604 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:40453
I20260620 07:08:26.015870 19604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:08:26.016486 19734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:40453 every 8 connection(s)
I20260620 07:08:26.025810 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 19604
I20260620 07:08:26.025929 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:08:26.026229 8288 external_mini_cluster.cc:1428] Setting key ae17e8e261cabddb0f0f5b1de678061e
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939305, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:08:26.051682 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.018903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39399 (local address 127.8.24.62:40879)
0620 07:08:26.030771 (+ 11868us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.030780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.036032 (+ 5252us) server_negotiation.cc:407] Connection header received
0620 07:08:26.037331 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.037335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.037407 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.037521 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:08:26.039183 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.040071 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.041809 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.042230 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.042305 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.047381 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:08:26.047385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:08:26.047388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:08:26.047419 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:08:26.049116 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.049704 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.049708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.049710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.049803 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.050230 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.050234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.050236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.050484 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:08:26.050558 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.051305 (+ 747us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.051489 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11780,"thread_start_us":102,"threads_started":1}
I20260620 07:08:26.055166 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.019313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40879 (local address 127.8.24.1:39399)
0620 07:08:26.035829 (+ 16516us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.035912 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.037069 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.037888 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.037899 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.038278 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:08:26.039007 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.039026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.040233 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.040236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.041674 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.041685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.042525 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.042528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.042721 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.045500 (+ 2779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:08:26.045522 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:08:26.047171 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:08:26.049240 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.049247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.049264 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.049591 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.049922 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.049924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.049927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.050119 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.050640 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:08:26.050647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:08:26.051003 (+ 356us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.052166 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.052541 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16232,"thread_start_us":125,"threads_started":1}
I20260620 07:08:26.056773 19735 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40879
I20260620 07:08:26.057032 19735 heartbeater.cc:464] Registering TS with master...
I20260620 07:08:26.057559 19735 heartbeater.cc:511] Master 127.8.24.62:40879 requested a full tablet report, sending...
I20260620 07:08:26.059263 19523 ts_manager.cc:194] Registered new tserver with Master: c40b360ab5f4428ea97bbf40fac26f77 (127.8.24.1:40453)
I20260620 07:08:26.060824 19523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:39399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:08:26.072258 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:50713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 07:08:26.206104 19743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:08:26.206508 19743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:08:26.206629 19743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:08:26.211321 19743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:08:26.211378 19743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:08:26.211479 19743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:08:26.217846 19743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.24.20:50713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:08:26.219182 19743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:08:26.220489 19743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:08:26.232674 19750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:26.232775 19752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:26.232690 19749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:26.234002 19743 server_base.cc:1034] running on GCE node
I20260620 07:08:26.234460 19743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:08:26.235180 19743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:08:26.236368 19743 hybrid_clock.cc:648] HybridClock initialized: now 1781939306236341 us; error 39 us; skew 500 ppm
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:08:26.242704 19743 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:08:26.244386 19743 webserver.cc:460] Webserver started at http://127.8.24.2:46133/ using document root <none> and password file <none>
I20260620 07:08:26.245163 19743 fs_manager.cc:359] Metadata directory not provided
I20260620 07:08:26.245256 19743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:08:26.245514 19743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:26.264541 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "433c304e923642d39569fe35930d299d"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "c344f1d9657785f5f6a27189db98576e"
server_key_iv: "a0fb0ae8181347306ff91512afee75ff"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.265259 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "433c304e923642d39569fe35930d299d"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "c344f1d9657785f5f6a27189db98576e"
server_key_iv: "a0fb0ae8181347306ff91512afee75ff"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.271252 19743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260620 07:08:26.310101 19759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.312314 19743 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260620 07:08:26.312471 19743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "433c304e923642d39569fe35930d299d"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "c344f1d9657785f5f6a27189db98576e"
server_key_iv: "a0fb0ae8181347306ff91512afee75ff"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.312613 19743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:08:26.361595 19743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:08:26.365432 19743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:08:26.365713 19743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:08:26.366474 19743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:08:26.367559 19743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:08:26.367671 19743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.367776 19743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:08:26.367832 19743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.377856 19743 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:36209
I20260620 07:08:26.379020 19743 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:08:26.384685 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 19743
I20260620 07:08:26.384805 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:08:26.385063 8288 external_mini_cluster.cc:1428] Setting key e96edbf34f5dafdfdc885ba3f1b27d44
I20260620 07:08:26.388335 19872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:36209 every 8 connection(s)
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:08:26.409332 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.391985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54439 (local address 127.8.24.62:40879)
0620 07:08:26.392112 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.392116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.392558 (+ 442us) server_negotiation.cc:407] Connection header received
0620 07:08:26.393927 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.393931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.393988 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.394045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:08:26.395717 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.396579 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.398253 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.398598 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.398680 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.401748 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:08:26.401751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:08:26.401754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:08:26.401784 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:08:26.403520 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.404115 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.404119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.404121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.404206 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.407958 (+ 3752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.407962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.407964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.408201 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:08:26.408290 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.409019 (+ 729us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.409158 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:08:26.411109 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.390803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40879 (local address 127.8.24.2:54439)
0620 07:08:26.392405 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.392464 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.393648 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.394463 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.394472 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.394820 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:08:26.395546 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.395569 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.396739 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.396742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.398114 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.398125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.398902 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.398907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.399109 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.399759 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:08:26.399779 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:08:26.401549 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:08:26.403645 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.403652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.403669 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.404000 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.404316 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.404320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.404322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.404499 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.408358 (+ 3859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:08:26.408365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:08:26.408708 (+ 343us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.409534 (+ 826us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.409884 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1377,"thread_start_us":162,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:08:26.413652 19873 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40879
I20260620 07:08:26.413944 19873 heartbeater.cc:464] Registering TS with master...
I20260620 07:08:26.414539 19873 heartbeater.cc:511] Master 127.8.24.62:40879 requested a full tablet report, sending...
I20260620 07:08:26.415678 19523 ts_manager.cc:194] Registered new tserver with Master: 433c304e923642d39569fe35930d299d (127.8.24.2:36209)
I20260620 07:08:26.416424 19523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:54439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:08:26.434514 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:50713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:08:26.588361 19878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:08:26.588771 19878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:08:26.588842 19878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:08:26.594846 19878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:08:26.594933 19878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:08:26.595021 19878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:08:26.602903 19878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.24.20:50713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:40879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:08:26.604321 19878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:08:26.605652 19878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:08:26.618497 19884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:08:26.618923 19883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:26.619611 19878 server_base.cc:1034] running on GCE node
W20260620 07:08:26.619933 19886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:08:26.620385 19878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:08:26.621134 19878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:08:26.622325 19878 hybrid_clock.cc:648] HybridClock initialized: now 1781939306622258 us; error 81 us; skew 500 ppm
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:08:26.625962 19878 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:08:26.627017 19878 webserver.cc:460] Webserver started at http://127.8.24.3:35847/ using document root <none> and password file <none>
I20260620 07:08:26.627760 19878 fs_manager.cc:359] Metadata directory not provided
I20260620 07:08:26.627840 19878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:08:26.628083 19878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:26.644307 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "4eec55bb5e4b44b792880abbab600570"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "b1bc51febfe2e46f2af5dc489b56251d"
server_key_iv: "54c13e6290b306f10ae32a6d682bf814"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.645052 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "4eec55bb5e4b44b792880abbab600570"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "b1bc51febfe2e46f2af5dc489b56251d"
server_key_iv: "54c13e6290b306f10ae32a6d682bf814"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.669304 19878 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.004s sys 0.000s
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:08:26.693063 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.694664 19878 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260620 07:08:26.694785 19878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "4eec55bb5e4b44b792880abbab600570"
format_stamp: "Formatted at 2026-06-20 07:08:26 on dist-test-slave-wjw3"
server_key: "b1bc51febfe2e46f2af5dc489b56251d"
server_key_iv: "54c13e6290b306f10ae32a6d682bf814"
server_key_version: "kuduclusterkey@0"
I20260620 07:08:26.694918 19878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:08:26.779814 19878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:08:26.794337 19878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:08:26.794605 19878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:08:26.795329 19878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:08:26.796432 19878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:08:26.796543 19878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.796640 19878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:08:26.796702 19878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:08:26.825078 19878 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:42621
I20260620 07:08:26.826409 19878 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:08:26.827061 20006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:42621 every 8 connection(s)
I20260620 07:08:26.832235 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 19878
I20260620 07:08:26.832343 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:08:26.832643 8288 external_mini_cluster.cc:1428] Setting key 9b967bd495c8ce4500dff662b17c0f37
Jun 20 07:08:26 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:08:26.853518 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.829214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:37057 (local address 127.8.24.62:40879)
0620 07:08:26.829357 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.829362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.840354 (+ 10992us) server_negotiation.cc:407] Connection header received
0620 07:08:26.841654 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.841658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.841716 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.841768 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:08:26.843470 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.844299 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.846001 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.846335 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.846405 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.849398 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:08:26.849402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:08:26.849405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:08:26.849434 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:08:26.851085 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.851716 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.851720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.851722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.851805 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.852205 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.852208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.852210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.852427 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:08:26.852502 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.853211 (+ 709us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.853334 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 07:08:26.855634 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.830091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:40879 (local address 127.8.24.3:37057)
0620 07:08:26.840183 (+ 10092us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.840250 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.841394 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.842223 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.842234 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.842589 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:08:26.843290 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.843309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.844458 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.844462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.845857 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.845867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.846598 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.846601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.846793 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.847502 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:08:26.847522 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:08:26.849205 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:08:26.851225 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.851232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.851250 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.851606 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.851907 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:08:26.851910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:08:26.851912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:08:26.852105 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:08:26.852564 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:08:26.852571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:08:26.852900 (+ 329us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.854139 (+ 1239us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.854493 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9799,"thread_start_us":119,"threads_started":1}
I20260620 07:08:26.857141 20007 heartbeater.cc:347] Connected to a master server at 127.8.24.62:40879
I20260620 07:08:26.857399 20007 heartbeater.cc:464] Registering TS with master...
I20260620 07:08:26.857970 20007 heartbeater.cc:511] Master 127.8.24.62:40879 requested a full tablet report, sending...
I20260620 07:08:26.859251 19523 ts_manager.cc:194] Registered new tserver with Master: 4eec55bb5e4b44b792880abbab600570 (127.8.24.3:42621)
I20260620 07:08:26.859995 19523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:37057
I20260620 07:08:26.862977 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:08:26.874634 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.865177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.8.24.62:40879)
0620 07:08:26.865284 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.865288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.865524 (+ 236us) server_negotiation.cc:407] Connection header received
0620 07:08:26.865699 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.865703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.865750 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.865799 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:08:26.866744 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.867603 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.868867 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.869244 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.869320 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.871124 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:08:26.871128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:08:26.871131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:08:26.871159 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:08:26.872798 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.873279 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.873282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.873285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.873363 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:08:26.873647 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:08:26.873650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:08:26.873652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:08:26.873882 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:08:26.873993 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.874342 (+ 349us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.874459 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:08:26.878522 19523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:08:26.883261 19523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:08:26.934341 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.919949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:42621 (local address 127.0.0.1:53274)
0620 07:08:26.922550 (+ 2601us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.922561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.922619 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.923487 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.923493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.923516 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:08:26.923642 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.923649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.926521 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.926525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.931747 (+ 5222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.931757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.933746 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.933750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.933916 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.933958 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.934065 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.934184 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2551,"thread_start_us":73,"threads_started":1}
I20260620 07:08:26.935973 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.916675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40453 (local address 127.0.0.1:33278)
0620 07:08:26.919711 (+ 3036us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.919763 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.919896 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.931527 (+ 11631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.931531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.931550 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:08:26.931704 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.931713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.933130 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.933133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.934304 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.934314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.935580 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.935584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.935680 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.935705 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.935775 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.935842 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2846,"thread_start_us":92,"threads_started":1}
I20260620 07:08:26.936410 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.917498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.8.24.1:40453)
0620 07:08:26.921589 (+ 4091us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.921597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.921622 (+ 25us) server_negotiation.cc:407] Connection header received
0620 07:08:26.921700 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.921705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.921934 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.922081 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:08:26.931854 (+ 9773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.932988 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.934459 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.935252 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.935350 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.935518 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.936168 (+ 650us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.936270 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3982,"thread_start_us":64,"threads_started":1}
I20260620 07:08:26.939837 19669 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc03501732f46908549a42723173116 (DEFAULT_TABLE table=test-table [id=178fa4a198034abe972a6edf414e16a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:08:26.941002 19669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc03501732f46908549a42723173116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:08:26.947562 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.918844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:36209 (local address 127.0.0.1:51922)
0620 07:08:26.922306 (+ 3462us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:26.922319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:26.922397 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:26.925042 (+ 2645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:26.925045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:26.925061 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:08:26.925150 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.925156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.929965 (+ 4809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.929969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.931234 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:26.931245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.936171 (+ 4926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.936175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:26.936269 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.936292 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:08:26.936360 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:08:26.936422 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3399,"thread_start_us":99,"threads_started":1}
I20260620 07:08:26.949738 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.920303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53274 (local address 127.8.24.3:42621)
0620 07:08:26.922840 (+ 2537us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.922850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.922871 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:08:26.922984 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.922990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.923177 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.923307 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:08:26.925266 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.926389 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.931908 (+ 5519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.933442 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.933543 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.933688 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.949459 (+ 15771us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.949598 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2439,"thread_start_us":149,"threads_started":1}
I20260620 07:08:26.952929 19939 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc03501732f46908549a42723173116 (DEFAULT_TABLE table=test-table [id=178fa4a198034abe972a6edf414e16a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:08:26.954257 19939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc03501732f46908549a42723173116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:08:26.965682 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:26.919217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.8.24.2:36209)
0620 07:08:26.924530 (+ 5313us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:26.924536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:26.924557 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:08:26.924621 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:26.924626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:26.924786 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:26.924917 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:08:26.928180 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.929822 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.934953 (+ 5131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:26.935769 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:26.935864 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:26.935999 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:26.963357 (+ 27358us) server_negotiation.cc:299] Negotiation successful
0620 07:08:26.963517 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5212,"thread_start_us":88,"threads_started":1}
I20260620 07:08:26.967957 20026 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Bootstrap starting.
I20260620 07:08:26.968389 20027 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Bootstrap starting.
I20260620 07:08:26.968880 19803 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc03501732f46908549a42723173116 (DEFAULT_TABLE table=test-table [id=178fa4a198034abe972a6edf414e16a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:08:26.969982 19803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc03501732f46908549a42723173116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:08:26.972896 20027 tablet_bootstrap.cc:654] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Neither blocks nor log segments found. Creating new log.
I20260620 07:08:26.973469 20026 tablet_bootstrap.cc:654] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Neither blocks nor log segments found. Creating new log.
I20260620 07:08:26.975175 20027 log.cc:824] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Log is configured to *not* fsync() on all Append() calls
I20260620 07:08:26.976146 20026 log.cc:824] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Log is configured to *not* fsync() on all Append() calls
I20260620 07:08:26.979022 20026 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: No bootstrap required, opened a new log
I20260620 07:08:26.979275 20026 ts_tablet_manager.cc:1397] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260620 07:08:26.983961 20026 raft_consensus.cc:348] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:26.984326 20026 raft_consensus.cc:374] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:08:26.984417 20026 raft_consensus.cc:729] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eec55bb5e4b44b792880abbab600570, State: Initialized, Role: FOLLOWER
I20260620 07:08:26.984975 20026 consensus_queue.cc:260] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:26.986004 20026 ts_tablet_manager.cc:1428] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260620 07:08:26.987717 20027 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: No bootstrap required, opened a new log
I20260620 07:08:26.987954 20027 ts_tablet_manager.cc:1397] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260620 07:08:26.992026 20027 raft_consensus.cc:348] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:26.992328 20027 raft_consensus.cc:374] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:08:26.992410 20027 raft_consensus.cc:729] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40b360ab5f4428ea97bbf40fac26f77, State: Initialized, Role: FOLLOWER
I20260620 07:08:26.992923 20027 consensus_queue.cc:260] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:26.993968 20027 ts_tablet_manager.cc:1428] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:08:26.994421 20007 heartbeater.cc:503] Master 127.8.24.62:40879 was elected leader, sending a full tablet report...
I20260620 07:08:26.994588 19735 heartbeater.cc:503] Master 127.8.24.62:40879 was elected leader, sending a full tablet report...
I20260620 07:08:27.002478 20030 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: Bootstrap starting.
I20260620 07:08:27.004956 20030 tablet_bootstrap.cc:654] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: Neither blocks nor log segments found. Creating new log.
I20260620 07:08:27.005846 20030 log.cc:824] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:08:27.008003 20030 tablet_bootstrap.cc:492] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: No bootstrap required, opened a new log
I20260620 07:08:27.008198 20030 ts_tablet_manager.cc:1397] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260620 07:08:27.012463 20030 raft_consensus.cc:348] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:27.012717 20030 raft_consensus.cc:374] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:08:27.012765 20030 raft_consensus.cc:729] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433c304e923642d39569fe35930d299d, State: Initialized, Role: FOLLOWER
I20260620 07:08:27.013412 20030 consensus_queue.cc:260] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:27.014587 20030 ts_tablet_manager.cc:1428] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 07:08:27.015009 19873 heartbeater.cc:503] Master 127.8.24.62:40879 was elected leader, sending a full tablet report...
W20260620 07:08:27.018005 19736 tablet.cc:2369] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:08:27.033613 20031 raft_consensus.cc:482] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:08:27.033833 20031 raft_consensus.cc:504] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:27.035252 20031 leader_election.cc:288] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40b360ab5f4428ea97bbf40fac26f77 (127.8.24.1:40453), 433c304e923642d39569fe35930d299d (127.8.24.2:36209)
I20260620 07:08:27.041503 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.035664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40453 (local address 127.8.24.3:46759)
0620 07:08:27.035779 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:27.035794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:27.035886 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:27.037579 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:27.037582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:27.037602 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:08:27.037714 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:27.037721 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.038881 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.038884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:27.040119 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:27.040131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.041085 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.041088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:27.041190 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:27.041214 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 07:08:27.041282 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 07:08:27.041362 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 07:08:27.042666 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.036920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46759 (local address 127.8.24.1:40453)
0620 07:08:27.037161 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:27.037166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:27.037181 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:08:27.037232 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:27.037235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:27.037280 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:27.037349 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:08:27.037998 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.038747 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.040266 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.040856 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.040963 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:27.041026 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:27.042479 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0620 07:08:27.042558 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260620 07:08:27.043179 19689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc03501732f46908549a42723173116" candidate_uuid: "4eec55bb5e4b44b792880abbab600570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40b360ab5f4428ea97bbf40fac26f77" is_pre_election: true
I20260620 07:08:27.043735 19689 raft_consensus.cc:2454] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eec55bb5e4b44b792880abbab600570 in term 0.
I20260620 07:08:27.044301 19895 leader_election.cc:302] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eec55bb5e4b44b792880abbab600570, c40b360ab5f4428ea97bbf40fac26f77; no voters:
I20260620 07:08:27.044653 20031 raft_consensus.cc:2781] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:08:27.044734 20031 raft_consensus.cc:482] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:08:27.044773 20031 raft_consensus.cc:3037] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:08:27.046056 20031 raft_consensus.cc:504] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:27.046389 20031 leader_election.cc:288] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [CANDIDATE]: Term 1 election: Requested vote from peers c40b360ab5f4428ea97bbf40fac26f77 (127.8.24.1:40453), 433c304e923642d39569fe35930d299d (127.8.24.2:36209)
I20260620 07:08:27.046893 19689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc03501732f46908549a42723173116" candidate_uuid: "4eec55bb5e4b44b792880abbab600570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40b360ab5f4428ea97bbf40fac26f77"
I20260620 07:08:27.047000 19689 raft_consensus.cc:3037] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:08:27.048173 19689 raft_consensus.cc:2454] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eec55bb5e4b44b792880abbab600570 in term 1.
I20260620 07:08:27.048535 19895 leader_election.cc:302] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eec55bb5e4b44b792880abbab600570, c40b360ab5f4428ea97bbf40fac26f77; no voters:
I20260620 07:08:27.050140 20031 raft_consensus.cc:2781] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:08:27.050376 20031 raft_consensus.cc:686] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 1 LEADER]: Becoming Leader. State: Replica: 4eec55bb5e4b44b792880abbab600570, State: Running, Role: LEADER
I20260620 07:08:27.050710 20031 consensus_queue.cc:237] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } }
I20260620 07:08:27.052110 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.035939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:36209 (local address 127.8.24.3:40433)
0620 07:08:27.041651 (+ 5712us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:27.041665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:08:27.041738 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:08:27.048558 (+ 6820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:08:27.048561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:08:27.048582 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:08:27.048670 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:27.048675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.049668 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.049672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:27.050707 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:08:27.050717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.051736 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.051740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:08:27.051819 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:27.051839 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 07:08:27.051901 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:08:27.051969 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5659,"thread_start_us":619,"threads_started":1}
I20260620 07:08:27.052546 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.037043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40433 (local address 127.8.24.2:36209)
0620 07:08:27.037519 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:27.037524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:27.041892 (+ 4368us) server_negotiation.cc:407] Connection header received
0620 07:08:27.041945 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:27.041948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:27.041998 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:27.042056 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:08:27.048788 (+ 6732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.049532 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.050886 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.051501 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.051595 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:27.051658 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:27.052342 (+ 684us) server_negotiation.cc:299] Negotiation successful
0620 07:08:27.052422 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387}
I20260620 07:08:27.053504 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc03501732f46908549a42723173116" candidate_uuid: "4eec55bb5e4b44b792880abbab600570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433c304e923642d39569fe35930d299d" is_pre_election: true
I20260620 07:08:27.053779 19826 raft_consensus.cc:2454] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eec55bb5e4b44b792880abbab600570 in term 0.
I20260620 07:08:27.054420 19522 catalog_manager.cc:5495] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eec55bb5e4b44b792880abbab600570 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "4eec55bb5e4b44b792880abbab600570" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eec55bb5e4b44b792880abbab600570" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42621 } health_report { overall_health: HEALTHY } } }
I20260620 07:08:27.053514 19827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc03501732f46908549a42723173116" candidate_uuid: "4eec55bb5e4b44b792880abbab600570" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433c304e923642d39569fe35930d299d"
I20260620 07:08:27.055768 19827 raft_consensus.cc:3037] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:08:27.057009 19827 raft_consensus.cc:2454] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eec55bb5e4b44b792880abbab600570 in term 1.
I20260620 07:08:27.080586 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.076018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.8.24.3:42621)
0620 07:08:27.076275 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 07:08:27.076281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:08:27.076299 (+ 18us) server_negotiation.cc:407] Connection header received
0620 07:08:27.076393 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:08:27.076396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:08:27.076471 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:08:27.076584 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:08:27.076850 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.077610 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.078849 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:08:27.079179 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:08:27.079283 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:08:27.079619 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:08:27.080168 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:08:27.080299 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 07:08:27.080365 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 07:08:27.080436 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Jun 20 07:08:27 dist-test-slave-wjw3 krb5kdc[19047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
W20260620 07:08:27.081743 20008 tablet.cc:2369] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:08:27.087881 19827 raft_consensus.cc:1261] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 1 FOLLOWER]: Refusing update from remote peer 4eec55bb5e4b44b792880abbab600570: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:08:27.089398 20036 consensus_queue.cc:1035] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [LEADER]: Connected to new peer: Peer: permanent_uuid: "433c304e923642d39569fe35930d299d" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:08:27.096347 19689 raft_consensus.cc:1261] T 5dc03501732f46908549a42723173116 P c40b360ab5f4428ea97bbf40fac26f77 [term 1 FOLLOWER]: Refusing update from remote peer 4eec55bb5e4b44b792880abbab600570: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:08:27.097163 20036 consensus_queue.cc:1035] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40b360ab5f4428ea97bbf40fac26f77" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:08:27.108006 20040 mvcc.cc:205] Tried to move back new op lower bound from 7298823401821274112 to 7298823401682944000. Current Snapshot: MvccSnapshot[applied={T|T < 7298823401821274112}]
I20260620 07:08:27.116309 19523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:08:27.116489 19523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:08:27.118929 20041 mvcc.cc:205] Tried to move back new op lower bound from 7298823401821274112 to 7298823401682944000. Current Snapshot: MvccSnapshot[applied={T|T < 7298823401821274112}]
I20260620 07:08:27.119127 19523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29562d197010470c9df4bc84ed91d271: Sending DeleteTablet for 3 replicas of tablet 5dc03501732f46908549a42723173116
I20260620 07:08:27.119546 20043 mvcc.cc:205] Tried to move back new op lower bound from 7298823401821274112 to 7298823401682944000. Current Snapshot: MvccSnapshot[applied={T|T < 7298823401821274112}]
I20260620 07:08:27.119912 19669 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc03501732f46908549a42723173116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:08:27 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:33278
I20260620 07:08:27.121040 19939 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc03501732f46908549a42723173116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:08:27 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:53274
I20260620 07:08:27.121331 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 19604
I20260620 07:08:27.120401 19803 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc03501732f46908549a42723173116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:08:27 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:51922
I20260620 07:08:27.129878 20051 tablet_replica.cc:330] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: stopping tablet replica
I20260620 07:08:27.130288 20051 raft_consensus.cc:2229] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:08:27.130666 20051 raft_consensus.cc:2258] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:08:27.134523 20051 ts_tablet_manager.cc:1905] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:08:27.137200 20051 ts_tablet_manager.cc:1918] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260620 07:08:27.143833 19508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:40453 (error 108)
I20260620 07:08:27.144066 20052 tablet_replica.cc:330] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d: stopping tablet replica
I20260620 07:08:27.144372 20052 raft_consensus.cc:2229] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:08:27.144695 20052 raft_consensus.cc:2258] T 5dc03501732f46908549a42723173116 P 433c304e923642d39569fe35930d299d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:08:27.144958 20051 log.cc:1196] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/5dc03501732f46908549a42723173116
I20260620 07:08:27.145015 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 19743
I20260620 07:08:27.145857 20051 ts_tablet_manager.cc:1939] T 5dc03501732f46908549a42723173116 P 4eec55bb5e4b44b792880abbab600570: Deleting consensus metadata
I20260620 07:08:27.147037 19507 catalog_manager.cc:4841] TS 4eec55bb5e4b44b792880abbab600570 (127.8.24.3:42621): tablet 5dc03501732f46908549a42723173116 (table test-table [id=178fa4a198034abe972a6edf414e16a0]) successfully deleted
I20260620 07:08:27.156035 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.155744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:40453 (local address 127.0.0.1:33290)
0620 07:08:27.155853 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:27.155936 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 07:08:27.156368 19508 catalog_manager.cc:4584] TS c40b360ab5f4428ea97bbf40fac26f77 (127.8.24.1:40453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5dc03501732f46908549a42723173116: Network error: Client connection negotiation failed: client connection to 127.8.24.1:40453: connect: Connection refused (error 111)
I20260620 07:08:27.156847 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:08:27.156134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:36209 (local address 127.0.0.1:51928)
0620 07:08:27.156726 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0620 07:08:27.156776 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":15}
I20260620 07:08:27.158874 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 19878
I20260620 07:08:27.170063 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 19491
I20260620 07:08:27.179208 8288 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 07:08:27.512730 8288 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 07:08:27.512850 8288 mini_ranger.cc:66] Stopping Ranger...
I20260620 07:08:27.846701 8288 mini_ranger.cc:68] Stopped Ranger
2026-06-20 07:08:27.846 UTC [19077] LOG: received smart shutdown request
2026-06-20 07:08:27.849 UTC [19077] LOG: background worker "logical replication launcher" (PID 19086) exited with exit code 1
2026-06-20 07:08:27.849 UTC [19081] LOG: shutting down
2026-06-20 07:08:27.964 UTC [19077] LOG: database system is shut down
2026-06-20T07:08:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44455 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:08:28 dist-test-slave-wjw3 krb5kdc[20063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:08:28 dist-test-slave-wjw3 krb5kdc[20063](info): set up 2 sockets
Jun 20 07:08:28 dist-test-slave-wjw3 krb5kdc[20063](info): commencing operation
krb5kdc: starting...
W20260620 07:08:28.995292 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:08:29 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:08:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangeradmin_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangerlookup_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
I20260620 07:08:29.141620 8288 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksJiSS1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/postgres -l logfile start
2026-06-20 07:08:31.283 UTC [20093] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 07:08:31.283 UTC [20093] LOG: listening on IPv4 address "127.8.24.20", port 49507
2026-06-20 07:08:31.287 UTC [20093] LOG: listening on Unix socket "/tmp/.s.PGSQL.49507"
2026-06-20 07:08:31.313 UTC [20096] LOG: database system was shut down at 2026-06-20 07:08:30 UTC
2026-06-20 07:08:31.316 UTC [20093] LOG: database system is ready to accept connections
I20260620 07:08:32.191058 8288 mini_postgres.cc:96] Postgres bound to 49507
2026-06-20 07:08:32.196 UTC [20104] FATAL: database "slave" does not exist
127.8.24.20:49507 - accepting connections
I20260620 07:08:32.197108 8288 mini_ranger.cc:161] Starting Ranger...
I20260620 07:08:32.210572 8288 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 07:08:32.604008 8288 mini_ranger.cc:87] Created ranger Postgres database
I20260620 07:08:32.604089 8288 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin
2026-06-20 07:08:32,864 [I] DB FLAVOR :POSTGRES
2026-06-20 07:08:32,864 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 07:08:32,864 [I] Checking connection..
2026-06-20 07:08:32,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 07:08:33,085 [I] Checking connection passed.
2026-06-20 07:08:33,085 [I] --------- Verifying version history table ---------
2026-06-20 07:08:33,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:08:33,335 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 07:08:33,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:08:33,569 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 07:08:33,570 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 07:08:33,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 07:08:33.803 UTC [20168] WARNING: there is no transaction in progress
2026-06-20 07:08:33,808 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 07:08:33,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 07:08:34,054 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 07:08:34,054 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 07:08:34,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 07:08:34,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 07:08:34,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wjw3.c.gcp-upstream.internal','N') ;"
2026-06-20 07:08:34,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 07:08:34,965 [I] Table x_portal_user does not exist in database ranger
2026-06-20 07:08:34,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 07:08:35,214 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 07:08:35,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 07:08:35,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 07:08:35,694 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 07:08:35,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 07:08:36.555 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.569 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.583 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.596 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.610 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.625 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.672 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.678 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.689 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.698 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.707 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.718 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.727 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.732 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.737 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:36.742 UTC [20276] WARNING: there is no transaction in progress
2026-06-20 07:08:37,259 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 07:08:37,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 07:08:37,504 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 07:08:37,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 07:08:37,750 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 07:08:37,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 07:08:38,000 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 07:08:38,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 07:08:38,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 07:08:38,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wjw3.c.gcp-upstream.internal';"
2026-06-20 07:08:38,729 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 07:08:38,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 07:08:38,976 [I] DB_PATCHES have already been applied
I20260620 07:08:38.981843 8288 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksJiSS1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 07:08:38.981904 8288 mini_ranger.cc:193] Using host: 127.8.24.20
I20260620 07:08:38.985136 8288 mini_ranger.cc:227] Ranger admin URL: http://127.8.24.20:35533
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 7:08:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 07:08:39.973644 8288 mini_ranger.cc:160] Time spent starting Ranger: real 7.777s user 0.000s sys 0.008s
Jun 20, 2026 7:08:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 7:08:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 7:08:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 7:08:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 7:08:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 07:09:00.047752 8288 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.24.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.24.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.24.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/rangerkms_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.24.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.24.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.24.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/HTTP_127.8.24.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 07:09:00 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 07:09:00.122063 8288 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 07:09:00.130666 8288 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 07:09:00.847220 8288 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 07:09:00.847286 8288 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms
I20260620 07:09:00.847304 8288 mini_ranger_kms.cc:227] Using postgres at 127.8.24.20:49507
2026-06-20 07:09:00,998 [I] DB FLAVOR :POSTGRES
2026-06-20 07:09:00,998 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 07:09:00,998 [I] Checking connection
2026-06-20 07:09:00,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 07:09:01,237 [I] connection success
2026-06-20 07:09:01,237 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 07:09:01,237 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 07:09:01,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 07:09:01,463 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 07:09:01,463 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 07:09:01,464 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 07:09:01,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.24.20:49507/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 07:09:01,702 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 07:09:02.154238 8288 mini_ranger_kms.cc:310] Created kms service
I20260620 07:09:02.327409 8288 mini_ranger_kms.cc:326] Created kudu user
I20260620 07:09:02.367463 8288 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 07:09:02 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:02.753257 8288 mini_ranger_kms.cc:384] Added ranger policy
I20260620 07:09:02.753412 8288 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksJiSS1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksJiSS1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 07:09:02.753433 8288 mini_ranger_kms.cc:240] Using host: 127.8.24.20
I20260620 07:09:02.757261 8288 mini_ranger_kms.cc:276] Ranger KMS PID: 20456
I20260620 07:09:02.757344 8288 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.24.20:43893
Jun 20, 2026 7:09:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 7:09:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 7:09:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 7:09:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 7:09:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 7:09:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 07:09:04 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781939344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:04.803289 8288 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.681s user 0.003s sys 0.133s
I20260620 07:09:04.803406 8288 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 07:09:04.803498 8288 mini_ranger_kms.cc:401] 127.8.24.20:43893/kms/v1/keys
Jun 20 07:09:05 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Jun 20 07:09:05 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.24.20@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
Jun 20 07:09:07 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:09:07.627691 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39885
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39885
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:07.735397 20507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:07.735677 20507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:07.735718 20507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:07.739717 20507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:07.739771 20507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:07.739790 20507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:07.739820 20507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:07.739845 20507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:07.745114 20507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.24.20:43893/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39885
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:07.746099 20507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:07.747079 20507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:07.755285 20512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:07.755280 20507 server_base.cc:1034] running on GCE node
W20260620 07:09:07.755262 20515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:07.755319 20513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:07.756047 20507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:07.757191 20507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:07.758350 20507 hybrid_clock.cc:648] HybridClock initialized: now 1781939347758336 us; error 39 us; skew 500 ppm
Jun 20 07:09:07 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939347, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:07.760865 20507 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:07.761674 20507 webserver.cc:460] Webserver started at http://127.8.24.62:45165/ using document root <none> and password file <none>
I20260620 07:09:07.762199 20507 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:07.762256 20507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:07.762436 20507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:09:07 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939347, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:07.816533 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "cbe14072800f473292c9c7a406749f73"
format_stamp: "Formatted at 2026-06-20 07:09:07 on dist-test-slave-wjw3"
server_key: "0dd382348b461dd06b5112786070b069"
server_key_iv: "715eddb641b2c5843ecb11e8f6c9fab6"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:07.817131 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "cbe14072800f473292c9c7a406749f73"
format_stamp: "Formatted at 2026-06-20 07:09:07 on dist-test-slave-wjw3"
server_key: "0dd382348b461dd06b5112786070b069"
server_key_iv: "715eddb641b2c5843ecb11e8f6c9fab6"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:07.820875 20507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 07:09:07.835736 20522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:07.837515 20507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 07:09:07.837656 20507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "cbe14072800f473292c9c7a406749f73"
format_stamp: "Formatted at 2026-06-20 07:09:07 on dist-test-slave-wjw3"
server_key: "0dd382348b461dd06b5112786070b069"
server_key_iv: "715eddb641b2c5843ecb11e8f6c9fab6"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:07.837796 20507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:07.869665 20507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:07.872068 20507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:07.872219 20507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:07.878868 20507 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39885
I20260620 07:09:07.878890 20574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39885 every 8 connection(s)
I20260620 07:09:07.879779 20507 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:07.882380 20575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:07.884691 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 20507
I20260620 07:09:07.884824 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:09:07.885131 8288 external_mini_cluster.cc:1428] Setting key 27f9a81ea16c37fa417b38524a5a9a43
I20260620 07:09:07.888259 20575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Bootstrap starting.
I20260620 07:09:07.890690 20575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:07.891531 20575 log.cc:824] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:07.893329 20575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: No bootstrap required, opened a new log
Jun 20 07:09:07 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:07.897434 20575 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:07.897703 20575 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:07.897789 20575 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe14072800f473292c9c7a406749f73, State: Initialized, Role: FOLLOWER
I20260620 07:09:07.898255 20575 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:07.898393 20575 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:07.898454 20575 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:07.898551 20575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:07.898708 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:07.886555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.8.24.62:39885)
0620 07:09:07.886929 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:07.886945 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:07.886981 (+ 36us) server_negotiation.cc:407] Connection header received
0620 07:09:07.887636 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:07.887654 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:07.888000 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:07.888274 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:07.889118 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:07.890113 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:07.891239 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:07.891804 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:07.891953 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:07.894246 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:07.894252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:07.894267 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:07.894296 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:07.895997 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:07.896385 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:07.896390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:07.896396 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:07.896523 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:07.896731 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:07.896734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:07.896736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:07.897212 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:07.897391 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:07.897561 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 07:09:07.897816 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
I20260620 07:09:07.899411 20575 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:07.899748 20575 leader_election.cc:302] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe14072800f473292c9c7a406749f73; no voters:
I20260620 07:09:07.900039 20575 leader_election.cc:288] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:09:07.900120 20580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:07.900308 20580 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 LEADER]: Becoming Leader. State: Replica: cbe14072800f473292c9c7a406749f73, State: Running, Role: LEADER
I20260620 07:09:07.900650 20580 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:07.901183 20575 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:07.902266 20582 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe14072800f473292c9c7a406749f73. Latest consensus state: current_term: 1 leader_uuid: "cbe14072800f473292c9c7a406749f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } } }
I20260620 07:09:07.902283 20581 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe14072800f473292c9c7a406749f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } } }
I20260620 07:09:07.902385 20582 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:07.902390 20581 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:07.902724 20589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:07.905671 20589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:07.906132 20575 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 07:09:07.914937 20589 catalog_manager.cc:1347] Generated new cluster ID: 935254330e4f410daf813009f13f1345
I20260620 07:09:07.915023 20589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:07.929013 20589 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:09:07.930260 20589 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:07.937076 20589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Generated new TSK 0
I20260620 07:09:07.938109 20589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:09:08.596633 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:08.732617 20620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:08.732857 20620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:08.732913 20620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:08.736910 20620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:08.736979 20620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:08.737058 20620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:08.743245 20620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:08.744343 20620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:08.745393 20620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:08.754575 20630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:08.754649 20627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:08.754832 20628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:08.755345 20620 server_base.cc:1034] running on GCE node
I20260620 07:09:08.755764 20620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:08.756369 20620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:08.757539 20620 hybrid_clock.cc:648] HybridClock initialized: now 1781939348757513 us; error 43 us; skew 500 ppm
Jun 20 07:09:08 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:08.760327 20620 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:08.761423 20620 webserver.cc:460] Webserver started at http://127.8.24.1:45015/ using document root <none> and password file <none>
I20260620 07:09:08.762115 20620 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:08.762205 20620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:08.762426 20620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:09:08 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:08.778254 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "8bf6a4dc0a2e409ca3934f21877841a4"
format_stamp: "Formatted at 2026-06-20 07:09:08 on dist-test-slave-wjw3"
server_key: "cb79b17837d88f20b2bf238edb5b97f8"
server_key_iv: "b1af396435db6d4b426848e9e98dd8ed"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:08.778939 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "8bf6a4dc0a2e409ca3934f21877841a4"
format_stamp: "Formatted at 2026-06-20 07:09:08 on dist-test-slave-wjw3"
server_key: "cb79b17837d88f20b2bf238edb5b97f8"
server_key_iv: "b1af396435db6d4b426848e9e98dd8ed"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:08.790120 20620 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.000s
I20260620 07:09:08.809646 20637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:08.811489 20620 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260620 07:09:08.811635 20620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "8bf6a4dc0a2e409ca3934f21877841a4"
format_stamp: "Formatted at 2026-06-20 07:09:08 on dist-test-slave-wjw3"
server_key: "cb79b17837d88f20b2bf238edb5b97f8"
server_key_iv: "b1af396435db6d4b426848e9e98dd8ed"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:08.811866 20620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:08.847101 20620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:08.854951 20620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:08.855208 20620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:08.856029 20620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:08.857112 20620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:08.857213 20620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:08.857327 20620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:08.857401 20620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:08.867302 20620 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45979
I20260620 07:09:08.872174 20620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:08.870978 20752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45979 every 8 connection(s)
I20260620 07:09:08.878188 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 20620
I20260620 07:09:08.878309 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:09:08.878621 8288 external_mini_cluster.cc:1428] Setting key e1539b521df2a50a989509a4f171bdd2
Jun 20 07:09:08 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:08.890617 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:08.875167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:55977 (local address 127.8.24.62:39885)
0620 07:09:08.875566 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:08.875574 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:08.877059 (+ 1485us) server_negotiation.cc:407] Connection header received
0620 07:09:08.878362 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:08.878366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:08.878425 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:08.878497 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:08.880313 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:08.881187 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:08.882902 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:08.883264 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:08.883334 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:08.886314 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:08.886318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:08.886321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:08.886348 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:08.888172 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:08.888790 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:08.888793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:08.888795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:08.888886 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:08.889292 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:08.889296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:08.889298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:08.889536 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:08.889624 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:08.890299 (+ 675us) server_negotiation.cc:299] Negotiation successful
0620 07:09:08.890426 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":109,"threads_started":1}
I20260620 07:09:08.892376 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:08.875689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.1:55977)
0620 07:09:08.876893 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:08.876949 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:08.878047 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:08.879102 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:08.879112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:08.879497 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:08.880156 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:08.880176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:08.881345 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:08.881348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:08.882770 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:08.882781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:08.883570 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:08.883574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:08.883770 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:08.884422 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:08.884443 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:08.886127 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:08.888308 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:08.888314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:08.888329 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:08.888670 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:08.888994 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:08.888996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:08.888999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:08.889178 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:08.889694 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:08.889700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:08.890022 (+ 322us) client_negotiation.cc:769] Sending connection context
0620 07:09:08.891042 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0620 07:09:08.891365 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":115,"threads_started":1}
I20260620 07:09:08.894207 20753 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
I20260620 07:09:08.894434 20753 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:08.894974 20753 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:08.896683 20539 ts_manager.cc:194] Registered new tserver with Master: 8bf6a4dc0a2e409ca3934f21877841a4 (127.8.24.1:45979)
I20260620 07:09:08.898181 20539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:55977
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:09:08.919787 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:09:08 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781939348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 07:09:09.076864 20759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:09.077346 20759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:09.077436 20759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:09.083495 20759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:09.083560 20759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:09.083622 20759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:09.090106 20759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:09.091195 20759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:09.092614 20759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:09.115554 20766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:09.115679 20765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:09.115554 20768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:09.116020 20759 server_base.cc:1034] running on GCE node
I20260620 07:09:09.116541 20759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:09.117210 20759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:09.118440 20759 hybrid_clock.cc:648] HybridClock initialized: now 1781939349118414 us; error 43 us; skew 500 ppm
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:09.121147 20759 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:09.122062 20759 webserver.cc:460] Webserver started at http://127.8.24.2:32911/ using document root <none> and password file <none>
I20260620 07:09:09.122605 20759 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:09.122668 20759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:09.122861 20759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:09.141381 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "943fe265bb50443583a025c519c07181"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "caaaa3fcab31d3628727c4fb8e624207"
server_key_iv: "1269f541c00b2b89ed433a0c7d13a9e9"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.142030 20759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "943fe265bb50443583a025c519c07181"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "caaaa3fcab31d3628727c4fb8e624207"
server_key_iv: "1269f541c00b2b89ed433a0c7d13a9e9"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.146036 20759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 07:09:09.171617 20775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.173448 20759 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260620 07:09:09.173599 20759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "943fe265bb50443583a025c519c07181"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "caaaa3fcab31d3628727c4fb8e624207"
server_key_iv: "1269f541c00b2b89ed433a0c7d13a9e9"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.173744 20759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:09.203334 20759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:09.207005 20759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:09.207206 20759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:09.207903 20759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:09.208917 20759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:09.208981 20759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.209033 20759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:09.209062 20759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.219461 20759 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39957
I20260620 07:09:09.219514 20888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39957 every 8 connection(s)
I20260620 07:09:09.220634 20759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:09.222363 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 20759
I20260620 07:09:09.222491 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:09:09.222770 8288 external_mini_cluster.cc:1428] Setting key e08089d6811bf948ad0deed1a448682d
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:09.237092 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:09.224284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:46159 (local address 127.8.24.62:39885)
0620 07:09:09.224804 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:09.224808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:09.224824 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:09:09.224959 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:09.224963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:09.225007 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:09.225052 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:09.226721 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.227566 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.229349 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.229727 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.229839 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:09.232877 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:09.232881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:09.232884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:09.232916 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:09.234623 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:09.235257 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:09.235261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:09.235263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:09.235354 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:09.235779 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:09.235783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:09.235785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:09.236002 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:09.236067 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:09.236762 (+ 695us) server_negotiation.cc:299] Negotiation successful
0620 07:09:09.236900 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447}
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:09:09.247498 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:09.223212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.2:46159)
0620 07:09:09.223732 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:09.223789 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:09.224732 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:09.225450 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:09.225459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:09.225807 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:09.226516 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:09.226538 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.227736 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.227740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:09.229209 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:09.229222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.229904 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.229908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:09.230123 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:09.230860 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:09.230884 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:09.232673 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:09.234761 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:09.234767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:09.234785 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:09.235129 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:09.235485 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:09.235488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:09.235490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:09.235675 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:09.236142 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:09.236160 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:09.236493 (+ 333us) client_negotiation.cc:769] Sending connection context
0620 07:09:09.237549 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0620 07:09:09.246627 (+ 9078us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":139,"threads_started":1}
I20260620 07:09:09.249102 20889 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:09:09.249435 20889 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:09.250087 20889 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:09.251302 20539 ts_manager.cc:194] Registered new tserver with Master: 943fe265bb50443583a025c519c07181 (127.8.24.2:39957)
I20260620 07:09:09.252099 20539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:46159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:09:09.266939 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939348, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
W20260620 07:09:09.427546 20894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:09.428385 20894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:09.428516 20894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:09.435137 20894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:09.435294 20894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:09.435451 20894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:09.443126 20894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:09.445422 20894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:09.446851 20894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:09.459679 20900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:09.460210 20899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:09.460814 20902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:09.461094 20894 server_base.cc:1034] running on GCE node
I20260620 07:09:09.461678 20894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:09.462476 20894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:09.465714 20894 hybrid_clock.cc:648] HybridClock initialized: now 1781939349465643 us; error 81 us; skew 500 ppm
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:09.469340 20894 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:09.470368 20894 webserver.cc:460] Webserver started at http://127.8.24.3:39797/ using document root <none> and password file <none>
I20260620 07:09:09.471007 20894 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:09.471071 20894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:09.471275 20894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:09.492878 20894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "e7968b42fad34d6b85d00bc125c9efba"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "e09261a26254877d6fab34b5acaf3be5"
server_key_iv: "db5d066d964886a6e80c0e8c8a195064"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.493669 20894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "e7968b42fad34d6b85d00bc125c9efba"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "e09261a26254877d6fab34b5acaf3be5"
server_key_iv: "db5d066d964886a6e80c0e8c8a195064"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.511549 20894 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.004s sys 0.000s
I20260620 07:09:09.535388 20909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.537177 20894 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260620 07:09:09.537300 20894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "e7968b42fad34d6b85d00bc125c9efba"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "e09261a26254877d6fab34b5acaf3be5"
server_key_iv: "db5d066d964886a6e80c0e8c8a195064"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.537428 20894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:09.570461 20894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:09.574100 20894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:09.574344 20894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:09.575040 20894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:09.576124 20894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:09.576215 20894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.576297 20894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:09.576347 20894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.595319 20894 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46485
I20260620 07:09:09.600740 20894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:09.603191 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 20894
I20260620 07:09:09.603331 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:09:09.603659 8288 external_mini_cluster.cc:1428] Setting key cab84b88487ead5745811e9f868511cf
I20260620 07:09:09.603930 21022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46485 every 8 connection(s)
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:09.624459 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:09.610692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:47865 (local address 127.8.24.62:39885)
0620 07:09:09.610798 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:09.610802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:09.610816 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:09:09.611545 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:09.611549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:09.611604 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:09.611652 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:09.613395 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.614284 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.616093 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.616492 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.616571 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:09.620009 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:09.620013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:09.620016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:09.620046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:09.621796 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:09.622408 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:09.622412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:09.622414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:09.622503 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:09.622907 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:09.622910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:09.622912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:09.623131 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:09.623225 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:09.624123 (+ 898us) server_negotiation.cc:299] Negotiation successful
0620 07:09:09.624274 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:09:09.626186 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:09.609684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.3:47865)
0620 07:09:09.610181 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:09.610238 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:09.611259 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:09.612116 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:09.612126 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:09.612504 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:09.613214 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:09.613236 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.614448 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.614452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:09.615951 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:09.615963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:09.617128 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:09.617132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:09.617340 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:09.618013 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:09.618035 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:09.619821 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:09.621925 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:09.621931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:09.621949 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:09.622290 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:09.622606 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:09.622610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:09.622612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:09.622799 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:09.623291 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:09.623297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:09.623821 (+ 524us) client_negotiation.cc:769] Sending connection context
0620 07:09:09.624728 (+ 907us) client_negotiation.cc:240] Negotiation successful
0620 07:09:09.625084 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":116,"threads_started":1}
I20260620 07:09:09.627568 21023 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
I20260620 07:09:09.627828 21023 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:09.628412 21023 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:09.629686 20538 ts_manager.cc:194] Registered new tserver with Master: e7968b42fad34d6b85d00bc125c9efba (127.8.24.3:46485)
I20260620 07:09:09.630478 20538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:47865
I20260620 07:09:09.641614 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:09.642374 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 20620
I20260620 07:09:09.651052 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 20759
I20260620 07:09:09.657792 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 20894
I20260620 07:09:09.664717 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 20507
I20260620 07:09:09.670432 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39885
--webserver_interface=127.8.24.62
--webserver_port=45165
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:39885
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:09.776327 21026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:09.776597 21026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:09.776660 21026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:09.780663 21026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:09.780725 21026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:09.780750 21026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:09.780772 21026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:09.780800 21026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:09.786149 21026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:39885
--ranger_config_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:39885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=45165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:09.787120 21026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:09.788093 21026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:09.796062 21034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:09.796167 21026 server_base.cc:1034] running on GCE node
W20260620 07:09:09.796094 21032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:09.796119 21031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:09.796795 21026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:09.797824 21026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:09.798981 21026 hybrid_clock.cc:648] HybridClock initialized: now 1781939349798963 us; error 29 us; skew 500 ppm
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:09.801462 21026 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:09.802297 21026 webserver.cc:460] Webserver started at http://127.8.24.62:45165/ using document root <none> and password file <none>
I20260620 07:09:09.802798 21026 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:09.802866 21026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 07:09:09 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:09.846963 21026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:09.849159 21042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:09.850374 21026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:09:09.850486 21026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "cbe14072800f473292c9c7a406749f73"
format_stamp: "Formatted at 2026-06-20 07:09:07 on dist-test-slave-wjw3"
server_key: "0dd382348b461dd06b5112786070b069"
server_key_iv: "715eddb641b2c5843ecb11e8f6c9fab6"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:09.851027 21026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:09.877879 21026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:09.880375 21026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:09.880545 21026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:09.887184 21026 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:39885
I20260620 07:09:09.887212 21094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:39885 every 8 connection(s)
I20260620 07:09:09.888207 21026 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:09.892279 21095 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:09:09.893864 21095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Bootstrap starting.
I20260620 07:09:09.896489 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21026
I20260620 07:09:09.896895 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:45979
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:39885
--webserver_port=45015
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:09.909633 21095 log.cc:824] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:09.916289 21095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:09.916625 21095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Bootstrap complete.
I20260620 07:09:09.920469 21095 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:09.920862 21095 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe14072800f473292c9c7a406749f73, State: Initialized, Role: FOLLOWER
I20260620 07:09:09.921298 21095 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:09.921414 21095 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:09.921475 21095 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:09.921563 21095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:09:09.923301 21095 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:09.923616 21095 leader_election.cc:302] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe14072800f473292c9c7a406749f73; no voters:
I20260620 07:09:09.923897 21095 leader_election.cc:288] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 07:09:09.923987 21099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 07:09:09.924188 21099 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [term 2 LEADER]: Becoming Leader. State: Replica: cbe14072800f473292c9c7a406749f73, State: Running, Role: LEADER
I20260620 07:09:09.924527 21099 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } }
I20260620 07:09:09.925055 21095 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:09.925860 21101 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe14072800f473292c9c7a406749f73. Latest consensus state: current_term: 2 leader_uuid: "cbe14072800f473292c9c7a406749f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } } }
I20260620 07:09:09.925977 21101 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:09.926286 21100 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cbe14072800f473292c9c7a406749f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe14072800f473292c9c7a406749f73" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 39885 } } }
I20260620 07:09:09.926342 21108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:09.926367 21100 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:09.929391 21108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:09.929873 21095 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 07:09:09.930078 21108 catalog_manager.cc:1259] Loaded cluster ID: 935254330e4f410daf813009f13f1345
I20260620 07:09:09.930125 21108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:09.932446 21108 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:09.933073 21108 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Loaded TSK: 0
I20260620 07:09:09.934265 21108 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:09:10.032611 21097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:10.033043 21097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:10.033165 21097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:10.039188 21097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:10.039321 21097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:10.039463 21097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:10.046972 21097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:45979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=45015
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:10.048321 21097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:10.049660 21097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:10.061812 21135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:10.061866 21133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:10.062105 21097 server_base.cc:1034] running on GCE node
W20260620 07:09:10.062235 21132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:10.062765 21097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:10.063618 21097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:10.064795 21097 hybrid_clock.cc:648] HybridClock initialized: now 1781939350064762 us; error 56 us; skew 500 ppm
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:10.068260 21097 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:10.069375 21097 webserver.cc:460] Webserver started at http://127.8.24.1:45015/ using document root <none> and password file <none>
I20260620 07:09:10.070123 21097 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:10.070281 21097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:10.087383 21097 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:10.090128 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.091456 21097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:10.091568 21097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "8bf6a4dc0a2e409ca3934f21877841a4"
format_stamp: "Formatted at 2026-06-20 07:09:08 on dist-test-slave-wjw3"
server_key: "cb79b17837d88f20b2bf238edb5b97f8"
server_key_iv: "b1af396435db6d4b426848e9e98dd8ed"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:10.092087 21097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:10.128567 21097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:10.133172 21097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:10.133431 21097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:10.134222 21097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:10.135458 21097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:10.135560 21097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.135677 21097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:10.135752 21097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.147192 21097 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:45979
I20260620 07:09:10.147305 21256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:45979 every 8 connection(s)
I20260620 07:09:10.148281 21097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:10.154454 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21097
I20260620 07:09:10.154939 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:39957
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:39885
--webserver_port=32911
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:10.167614 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.150491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.1:34327)
0620 07:09:10.151468 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:10.151508 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:10.152277 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:10.153377 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:10.153385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:10.153708 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:10.154286 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.154305 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.156112 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.156115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.157657 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.157668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.158791 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.158795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.158964 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.159640 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:10.159662 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:10.161366 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:10.163865 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.163873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.163887 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.164237 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.164650 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.164654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.164656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.164836 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.165654 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:10.165661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:10.165993 (+ 332us) client_negotiation.cc:769] Sending connection context
0620 07:09:10.166219 (+ 226us) client_negotiation.cc:240] Negotiation successful
0620 07:09:10.166552 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":120,"threads_started":1}
I20260620 07:09:10.169466 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.150340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34327 (local address 127.8.24.62:39885)
0620 07:09:10.150938 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:10.150959 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:10.151628 (+ 669us) server_negotiation.cc:407] Connection header received
0620 07:09:10.152698 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:10.152709 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:10.152999 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:10.153282 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:10.154474 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.155966 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.157792 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.158452 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.158594 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.161585 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:10.161592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:10.161610 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:10.161642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:10.163725 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.164374 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.164381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.164387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.164538 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.164946 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.164950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.164952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.165464 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:10.165595 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:10.168277 (+ 2682us) server_negotiation.cc:299] Negotiation successful
0620 07:09:10.168563 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":105,"threads_started":1}
I20260620 07:09:10.172674 21257 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
I20260620 07:09:10.172904 21257 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.173336 21257 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.175580 21257 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39885 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 07:09:10.175693 21257 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.175850 21257 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:10.176656 21257 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.176820 21257 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.177572 21257 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:09:10.312508 21261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:10.312808 21261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:10.312870 21261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:10.317024 21261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:10.317087 21261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:10.317165 21261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:10.322449 21261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:39957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=32911
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:10.323551 21261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:10.324612 21261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:10.335525 21267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:10.335740 21269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:10.335980 21261 server_base.cc:1034] running on GCE node
W20260620 07:09:10.337013 21266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:10.337530 21261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:10.338368 21261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:10.339625 21261 hybrid_clock.cc:648] HybridClock initialized: now 1781939350339554 us; error 88 us; skew 500 ppm
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:10.342374 21261 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:10.343287 21261 webserver.cc:460] Webserver started at http://127.8.24.2:32911/ using document root <none> and password file <none>
I20260620 07:09:10.343995 21261 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:10.344153 21261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:10.361617 21261 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 07:09:10.367828 21277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.369423 21261 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260620 07:09:10.369542 21261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "943fe265bb50443583a025c519c07181"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "caaaa3fcab31d3628727c4fb8e624207"
server_key_iv: "1269f541c00b2b89ed433a0c7d13a9e9"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:10.370237 21261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:10.402199 21261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:10.405800 21261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:10.406011 21261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:10.406776 21261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:10.407909 21261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:10.407977 21261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.408031 21261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:10.408064 21261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.419899 21261 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39957
I20260620 07:09:10.421308 21261 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:10.423000 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21261
I20260620 07:09:10.423516 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.24.20:43893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:46485
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:39885
--webserver_port=39797
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:10.426535 21391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39957 every 8 connection(s)
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:10.439085 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.426437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54477 (local address 127.8.24.62:39885)
0620 07:09:10.426575 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:10.426580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:10.427391 (+ 811us) server_negotiation.cc:407] Connection header received
0620 07:09:10.428753 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:10.428757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:10.428810 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:10.428902 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:10.430431 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.431227 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.432919 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.433260 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.433338 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.435898 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:10.435901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:10.435903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:10.435927 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:10.437187 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.437653 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.437655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.437656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.437728 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.438040 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.438042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.438044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.438212 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:10.438292 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:10.438798 (+ 506us) server_negotiation.cc:299] Negotiation successful
0620 07:09:10.438922 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:09:10.439963 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.425278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.2:54477)
0620 07:09:10.427229 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:10.427288 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:10.428493 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:10.429199 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:10.429208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:10.429569 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:10.430251 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.430273 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.431388 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.431392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.432781 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.432792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.433548 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.433551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.433691 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.434246 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:10.434263 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:10.435742 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:10.437292 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.437298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.437314 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.437555 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.437805 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.437808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.437810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.437949 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.438305 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:10.438312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:10.438566 (+ 254us) client_negotiation.cc:769] Sending connection context
0620 07:09:10.438728 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 07:09:10.439045 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"thread_start_us":160,"threads_started":1}
I20260620 07:09:10.441152 21392 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
I20260620 07:09:10.441376 21392 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.441845 21392 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.443222 21392 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39885 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 07:09:10.443336 21392 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.443527 21392 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:10.444336 21392 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.444491 21392 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.445159 21392 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:09:10.586722 21394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:10.587100 21394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:10.587170 21394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:10.593375 21394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:10.593534 21394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:10.593657 21394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:10.601603 21394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.24.20:43893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:46485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=39797
--tserver_master_addrs=127.8.24.62:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:10.603132 21394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:10.604540 21394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:10.616637 21400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:10.616760 21403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:10.616633 21401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:10.617040 21394 server_base.cc:1034] running on GCE node
I20260620 07:09:10.617455 21394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:10.618166 21394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:10.619413 21394 hybrid_clock.cc:648] HybridClock initialized: now 1781939350619384 us; error 43 us; skew 500 ppm
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:10.622191 21394 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:10.623121 21394 webserver.cc:460] Webserver started at http://127.8.24.3:39797/ using document root <none> and password file <none>
I20260620 07:09:10.623739 21394 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:10.623821 21394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:10.641099 21394 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 07:09:10.644146 21411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.645706 21394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:09:10.645833 21394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "e7968b42fad34d6b85d00bc125c9efba"
format_stamp: "Formatted at 2026-06-20 07:09:09 on dist-test-slave-wjw3"
server_key: "e09261a26254877d6fab34b5acaf3be5"
server_key_iv: "db5d066d964886a6e80c0e8c8a195064"
server_key_version: "kuduclusterkey@0"
I20260620 07:09:10.646552 21394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:10.662307 21394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:10.666290 21394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:10.666572 21394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:10.667340 21394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:10.668465 21394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:10.668569 21394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.668663 21394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:10.668723 21394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:10.679914 21394 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46485
I20260620 07:09:10.679941 21524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46485 every 8 connection(s)
I20260620 07:09:10.681280 21394 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:10.685729 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21394
Jun 20 07:09:10 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939350, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:10.696249 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.686383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.8.24.62:39885)
0620 07:09:10.687623 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:10.687635 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:10.687772 (+ 137us) server_negotiation.cc:407] Connection header received
0620 07:09:10.687907 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:10.687909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:10.687948 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:10.688031 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:10.688969 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.689531 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.690588 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.690941 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.691051 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.692615 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:10.692619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:10.692622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:10.692647 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:10.694363 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.695059 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.695063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.695065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.695143 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.695386 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.695389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.695391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.695598 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:10.695785 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:10.695947 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 07:09:10.696069 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":86,"threads_started":1}
I20260620 07:09:10.698220 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.685353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:55023 (local address 127.8.24.62:39885)
0620 07:09:10.685507 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:10.685512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:10.686729 (+ 1217us) server_negotiation.cc:407] Connection header received
0620 07:09:10.687662 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:10.687666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:10.687713 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:10.687759 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:10.689080 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.689846 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.691194 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.691548 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.691651 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.694037 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:10.694041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:10.694043 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:10.694085 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:10.695532 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.696212 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.696215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.696217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.696301 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:10.696733 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:10.696736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:10.696738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:10.696953 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:10.697024 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:10.698036 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0620 07:09:10.698121 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 07:09:10.699342 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:10.685793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:39885 (local address 127.8.24.3:55023)
0620 07:09:10.686562 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:10.686617 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:10.687411 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:10.688179 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:10.688186 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:10.688443 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:10.688919 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.688937 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.689982 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.689986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.691058 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:10.691068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:10.691703 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:10.691708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:10.691910 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:10.692505 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:10.692525 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:10.693873 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:10.695658 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.695664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.695680 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.696097 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.696412 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:10.696415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:10.696417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:10.696617 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:10.697345 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:10.697351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:10.697699 (+ 348us) client_negotiation.cc:769] Sending connection context
0620 07:09:10.697919 (+ 220us) client_negotiation.cc:240] Negotiation successful
0620 07:09:10.698307 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260620 07:09:10.701139 21525 heartbeater.cc:347] Connected to a master server at 127.8.24.62:39885
I20260620 07:09:10.701401 21525 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.701973 21525 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.703554 21525 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:39885 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 07:09:10.703684 21525 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.703857 21525 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:10.704705 21525 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:10.704874 21525 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
W20260620 07:09:10.707160 21525 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 07:09:11 dist-test-slave-wjw3 krb5kdc[20063](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781939351, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:11.177768 21257 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:11.178036 21257 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:11.178975 21059 ts_manager.cc:194] Registered new tserver with Master: 8bf6a4dc0a2e409ca3934f21877841a4 (127.8.24.1:45979)
I20260620 07:09:11.180210 21059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:34327
Jun 20 07:09:11 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939351, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:11.445374 21392 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:11.445662 21392 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:11.446384 21059 ts_manager.cc:194] Registered new tserver with Master: 943fe265bb50443583a025c519c07181 (127.8.24.2:39957)
I20260620 07:09:11.447186 21059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:54477
Jun 20 07:09:11 dist-test-slave-wjw3 krb5kdc[20063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939351, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for HTTP/127.8.24.20@KRBTEST.COM
I20260620 07:09:11.707408 21525 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:11.707738 21525 heartbeater.cc:511] Master 127.8.24.62:39885 requested a full tablet report, sending...
I20260620 07:09:11.708578 21059 ts_manager.cc:194] Registered new tserver with Master: e7968b42fad34d6b85d00bc125c9efba (127.8.24.3:46485)
I20260620 07:09:11.709342 21059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:55023
I20260620 07:09:11.714069 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:11.732368 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.715887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.8.24.62:39885)
0620 07:09:11.716877 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.716881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.716893 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:09:11.717586 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.717590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.717644 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.718539 (+ 895us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:11.718642 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.719508 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.720880 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.721258 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.721747 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.726775 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:11.726782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:11.726785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:11.726818 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:11.729051 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:11.729550 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:11.729553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:11.729555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:11.729642 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:11.730378 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:11.730381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:11.730383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:11.730965 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:11.731793 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.731867 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.731993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":89,"threads_started":1}
I20260620 07:09:11.737557 21059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:09:11.742127 21059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:09:11.768029 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.759063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39957 (local address 127.0.0.1:40246)
0620 07:09:11.760352 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:11.760365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:11.760449 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:11.761233 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:11.761236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:11.761252 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:11.761336 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.761341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.762713 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.762717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.765063 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.765073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.766306 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.766309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.767584 (+ 1275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.767631 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 07:09:11.767742 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 07:09:11.767875 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"mutex_wait_us":2333,"thread_start_us":91,"threads_started":1}
I20260620 07:09:11.768091 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.759767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.8.24.1:45979)
0620 07:09:11.763272 (+ 3505us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.763279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.763299 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:09:11.763364 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.763369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.763573 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.763698 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:11.764056 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.765126 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.766536 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.767300 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.767393 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.767544 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.767866 (+ 322us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.767954 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3409,"thread_start_us":89,"threads_started":1}
I20260620 07:09:11.768622 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.759383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45979 (local address 127.0.0.1:49130)
0620 07:09:11.762979 (+ 3596us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:11.762991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:11.763064 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:11.763839 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:11.763842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:11.763857 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:11.763942 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.763947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.765246 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.765249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.766400 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.766410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.767600 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.767604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.767706 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.767728 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:09:11.767789 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:09:11.768484 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3540,"mutex_wait_us":611,"thread_start_us":89,"threads_started":1}
I20260620 07:09:11.770967 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.759266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.8.24.2:39957)
0620 07:09:11.760685 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.760691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.760711 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:09:11.760779 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.760784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.760964 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.761095 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:11.761444 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.762592 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.765202 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.765991 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.766093 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.766242 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.770727 (+ 4485us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.770830 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1324,"thread_start_us":80,"threads_started":1}
I20260620 07:09:11.771659 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.753422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46485 (local address 127.0.0.1:57900)
0620 07:09:11.759725 (+ 6303us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:11.759781 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:11.759916 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:11.760969 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:11.760975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:11.760999 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:11.761139 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.761147 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.762541 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.762544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.763963 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.763975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.771215 (+ 7240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.771219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.771310 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.771335 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:09:11.771407 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 07:09:11.771536 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6096,"mutex_wait_us":22,"thread_start_us":94,"threads_started":1}
I20260620 07:09:11.771993 21172 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7a187a6fe74629802fe1d92350e396 (DEFAULT_TABLE table=test-table [id=601a2354a5184ab2a212603ff4696b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:11.773176 21172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7a187a6fe74629802fe1d92350e396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:11.774559 21325 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7a187a6fe74629802fe1d92350e396 (DEFAULT_TABLE table=test-table [id=601a2354a5184ab2a212603ff4696b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:11.775539 21325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7a187a6fe74629802fe1d92350e396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:11.782946 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.754201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.8.24.3:46485)
0620 07:09:11.760255 (+ 6054us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.760262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.760288 (+ 26us) server_negotiation.cc:407] Connection header received
0620 07:09:11.760365 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.760370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.760614 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.760775 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:11.761276 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.762433 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.766408 (+ 3975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.767201 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.767309 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.767474 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.782649 (+ 15175us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.782792 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5892,"thread_start_us":110,"threads_started":1}
I20260620 07:09:11.783742 21561 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Bootstrap starting.
I20260620 07:09:11.785859 21561 tablet_bootstrap.cc:654] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:11.786175 21440 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7a187a6fe74629802fe1d92350e396 (DEFAULT_TABLE table=test-table [id=601a2354a5184ab2a212603ff4696b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:11.786621 21561 log.cc:824] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:11.787171 21440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7a187a6fe74629802fe1d92350e396. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:11.787222 21562 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Bootstrap starting.
I20260620 07:09:11.789309 21562 tablet_bootstrap.cc:654] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:11.790048 21562 log.cc:824] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:11.796010 21565 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Bootstrap starting.
I20260620 07:09:11.798228 21565 tablet_bootstrap.cc:654] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:11.798440 21561 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: No bootstrap required, opened a new log
I20260620 07:09:11.798651 21561 ts_tablet_manager.cc:1397] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260620 07:09:11.799105 21565 log.cc:824] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:11.801373 21562 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: No bootstrap required, opened a new log
I20260620 07:09:11.801571 21562 ts_tablet_manager.cc:1397] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260620 07:09:11.802731 21561 raft_consensus.cc:348] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.802995 21561 raft_consensus.cc:374] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:11.803040 21561 raft_consensus.cc:729] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6a4dc0a2e409ca3934f21877841a4, State: Initialized, Role: FOLLOWER
I20260620 07:09:11.803582 21561 consensus_queue.cc:260] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.804615 21561 ts_tablet_manager.cc:1428] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260620 07:09:11.804963 21257 heartbeater.cc:503] Master 127.8.24.62:39885 was elected leader, sending a full tablet report...
I20260620 07:09:11.805626 21562 raft_consensus.cc:348] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.805891 21562 raft_consensus.cc:374] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:11.805946 21562 raft_consensus.cc:729] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943fe265bb50443583a025c519c07181, State: Initialized, Role: FOLLOWER
I20260620 07:09:11.806413 21562 consensus_queue.cc:260] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.807400 21562 ts_tablet_manager.cc:1428] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:09:11.807794 21392 heartbeater.cc:503] Master 127.8.24.62:39885 was elected leader, sending a full tablet report...
I20260620 07:09:11.809857 21565 tablet_bootstrap.cc:492] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: No bootstrap required, opened a new log
I20260620 07:09:11.810067 21565 ts_tablet_manager.cc:1397] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260620 07:09:11.814200 21565 raft_consensus.cc:348] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.814536 21565 raft_consensus.cc:374] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:11.814627 21565 raft_consensus.cc:729] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7968b42fad34d6b85d00bc125c9efba, State: Initialized, Role: FOLLOWER
I20260620 07:09:11.815352 21565 consensus_queue.cc:260] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.816213 21525 heartbeater.cc:503] Master 127.8.24.62:39885 was elected leader, sending a full tablet report...
I20260620 07:09:11.816493 21565 ts_tablet_manager.cc:1428] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260620 07:09:11.901175 21258 tablet.cc:2369] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:11.921913 21569 raft_consensus.cc:482] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:11.922178 21569 raft_consensus.cc:504] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.923635 21569 leader_election.cc:288] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 943fe265bb50443583a025c519c07181 (127.8.24.2:39957), 8bf6a4dc0a2e409ca3934f21877841a4 (127.8.24.1:45979)
W20260620 07:09:11.927951 21393 tablet.cc:2369] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:11.929164 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.924008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:35749 (local address 127.8.24.2:39957)
0620 07:09:11.924141 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.924146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.924399 (+ 253us) server_negotiation.cc:407] Connection header received
0620 07:09:11.924628 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.924631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.924687 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.924736 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:11.925199 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.926016 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.927746 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.928376 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.928453 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.928514 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.928956 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.929031 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 07:09:11.929836 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.923834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39957 (local address 127.8.24.3:35749)
0620 07:09:11.924300 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:11.924316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:11.924487 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:11.924882 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:11.924885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:11.924905 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:11.925062 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.925069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.926160 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.926163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.927602 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.927613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.928706 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.928710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.928813 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.928837 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 07:09:11.929600 (+ 763us) client_negotiation.cc:240] Negotiation successful
0620 07:09:11.929678 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260620 07:09:11.930577 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.924584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:53989 (local address 127.8.24.1:45979)
0620 07:09:11.924695 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.924699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.924952 (+ 253us) server_negotiation.cc:407] Connection header received
0620 07:09:11.925149 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.925152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.925202 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.925247 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:11.925715 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.926523 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.928787 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.929356 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.929436 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.929493 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.930390 (+ 897us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.930462 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:09:11.930588 21345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7a187a6fe74629802fe1d92350e396" candidate_uuid: "e7968b42fad34d6b85d00bc125c9efba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943fe265bb50443583a025c519c07181" is_pre_election: true
I20260620 07:09:11.930895 21345 raft_consensus.cc:2454] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7968b42fad34d6b85d00bc125c9efba in term 0.
I20260620 07:09:11.930956 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.924250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45979 (local address 127.8.24.3:53989)
0620 07:09:11.924869 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:11.924882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:11.925030 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:11.925436 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:11.925438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:11.925458 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:11.925577 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.925583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.926700 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.926703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.928647 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:11.928658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.930161 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.930165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:11.930261 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.930282 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:09:11.930750 (+ 468us) client_negotiation.cc:240] Negotiation successful
0620 07:09:11.930822 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":86,"threads_started":1}
I20260620 07:09:11.931407 21415 leader_election.cc:302] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943fe265bb50443583a025c519c07181, e7968b42fad34d6b85d00bc125c9efba; no voters:
I20260620 07:09:11.931737 21569 raft_consensus.cc:2781] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:11.931818 21569 raft_consensus.cc:482] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:11.931861 21569 raft_consensus.cc:3037] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:11.931916 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7a187a6fe74629802fe1d92350e396" candidate_uuid: "e7968b42fad34d6b85d00bc125c9efba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" is_pre_election: true
I20260620 07:09:11.932209 21204 raft_consensus.cc:2454] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7968b42fad34d6b85d00bc125c9efba in term 0.
I20260620 07:09:11.933151 21569 raft_consensus.cc:504] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.933521 21569 leader_election.cc:288] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [CANDIDATE]: Term 1 election: Requested vote from peers 943fe265bb50443583a025c519c07181 (127.8.24.2:39957), 8bf6a4dc0a2e409ca3934f21877841a4 (127.8.24.1:45979)
I20260620 07:09:11.933881 21345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7a187a6fe74629802fe1d92350e396" candidate_uuid: "e7968b42fad34d6b85d00bc125c9efba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943fe265bb50443583a025c519c07181"
I20260620 07:09:11.933947 21204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7a187a6fe74629802fe1d92350e396" candidate_uuid: "e7968b42fad34d6b85d00bc125c9efba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6a4dc0a2e409ca3934f21877841a4"
I20260620 07:09:11.934000 21345 raft_consensus.cc:3037] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:11.934072 21204 raft_consensus.cc:3037] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:11.935250 21345 raft_consensus.cc:2454] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7968b42fad34d6b85d00bc125c9efba in term 1.
I20260620 07:09:11.935348 21204 raft_consensus.cc:2454] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7968b42fad34d6b85d00bc125c9efba in term 1.
I20260620 07:09:11.935678 21415 leader_election.cc:302] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6a4dc0a2e409ca3934f21877841a4, e7968b42fad34d6b85d00bc125c9efba; no voters:
I20260620 07:09:11.935854 21569 raft_consensus.cc:2781] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 1 FOLLOWER]: Leader election won for term 1
W20260620 07:09:11.936014 21526 tablet.cc:2369] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:11.936077 21569 raft_consensus.cc:686] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 1 LEADER]: Becoming Leader. State: Replica: e7968b42fad34d6b85d00bc125c9efba, State: Running, Role: LEADER
I20260620 07:09:11.936381 21569 consensus_queue.cc:237] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } }
I20260620 07:09:11.939888 21058 catalog_manager.cc:5495] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba reported cstate change: term changed from 0 to 1, leader changed from <none> to e7968b42fad34d6b85d00bc125c9efba (127.8.24.3). New cstate: current_term: 1 leader_uuid: "e7968b42fad34d6b85d00bc125c9efba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7968b42fad34d6b85d00bc125c9efba" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 } health_report { overall_health: UNKNOWN } } }
I20260620 07:09:11.960510 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:11.955649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.8.24.3:46485)
0620 07:09:11.955761 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:11.955765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:11.956055 (+ 290us) server_negotiation.cc:407] Connection header received
0620 07:09:11.956191 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:11.956193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:11.956249 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:11.956339 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:11.956720 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.957487 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.958687 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:11.959008 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:11.959129 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:11.959518 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:11.960088 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:11.960208 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:11.960283 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 07:09:11.960348 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 07:09:11.966604 21204 raft_consensus.cc:1261] T 1a7a187a6fe74629802fe1d92350e396 P 8bf6a4dc0a2e409ca3934f21877841a4 [term 1 FOLLOWER]: Refusing update from remote peer e7968b42fad34d6b85d00bc125c9efba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:11.966643 21345 raft_consensus.cc:1261] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 1 FOLLOWER]: Refusing update from remote peer e7968b42fad34d6b85d00bc125c9efba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:11.967259 21569 consensus_queue.cc:1035] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [LEADER]: Connected to new peer: Peer: permanent_uuid: "943fe265bb50443583a025c519c07181" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:11.967495 21572 consensus_queue.cc:1035] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6a4dc0a2e409ca3934f21877841a4" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 45979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:11.980878 21583 mvcc.cc:205] Tried to move back new op lower bound from 7298823585648152576 to 7298823585534156800. Current Snapshot: MvccSnapshot[applied={T|T < 7298823585648152576}]
I20260620 07:09:11.983824 21059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:11.983989 21059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:11.986395 21059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbe14072800f473292c9c7a406749f73: Sending DeleteTablet for 3 replicas of tablet 1a7a187a6fe74629802fe1d92350e396
I20260620 07:09:11.987082 21440 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7a187a6fe74629802fe1d92350e396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:57900
I20260620 07:09:11.987200 21325 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7a187a6fe74629802fe1d92350e396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:40246
I20260620 07:09:11.987648 21172 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a7a187a6fe74629802fe1d92350e396 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:11 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:49130
I20260620 07:09:11.987854 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21097
I20260620 07:09:11.989259 21592 tablet_replica.cc:330] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: stopping tablet replica
I20260620 07:09:11.989552 21592 raft_consensus.cc:2229] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:11.989852 21592 raft_consensus.cc:2258] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:11.990238 21591 tablet_replica.cc:330] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: stopping tablet replica
I20260620 07:09:11.990548 21591 raft_consensus.cc:2229] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:09:11.990903 21591 raft_consensus.cc:2258] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:11.992399 21591 ts_tablet_manager.cc:1905] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:11.994328 21592 ts_tablet_manager.cc:1905] T 1a7a187a6fe74629802fe1d92350e396 P 943fe265bb50443583a025c519c07181: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:11.995150 21591 ts_tablet_manager.cc:1918] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:11.995501 21591 log.cc:1196] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/1a7a187a6fe74629802fe1d92350e396
W20260620 07:09:11.995782 21046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:45979 (error 108)
I20260620 07:09:11.995994 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21261
I20260620 07:09:11.996771 21591 ts_tablet_manager.cc:1939] T 1a7a187a6fe74629802fe1d92350e396 P e7968b42fad34d6b85d00bc125c9efba: Deleting consensus metadata
I20260620 07:09:11.998075 21043 catalog_manager.cc:4841] TS e7968b42fad34d6b85d00bc125c9efba (127.8.24.3:46485): tablet 1a7a187a6fe74629802fe1d92350e396 (table test-table [id=601a2354a5184ab2a212603ff4696b87]) successfully deleted
I20260620 07:09:12.002802 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21394
I20260620 07:09:12.003531 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:12.003172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:45979 (local address 127.0.0.1:49134)
0620 07:09:12.003352 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:12.003417 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260620 07:09:12.003706 21046 catalog_manager.cc:4584] TS 8bf6a4dc0a2e409ca3934f21877841a4 (127.8.24.1:45979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a7a187a6fe74629802fe1d92350e396: Network error: Client connection negotiation failed: client connection to 127.8.24.1:45979: connect: Connection refused (error 111)
I20260620 07:09:12.004909 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:12.004182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39957 (local address 127.0.0.1:40258)
0620 07:09:12.004791 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:12.004841 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:39957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539}
I20260620 07:09:12.008579 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21026
I20260620 07:09:12.014496 8288 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 07:09:12.348023 8288 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 07:09:12.348137 8288 mini_ranger.cc:66] Stopping Ranger...
I20260620 07:09:12.681459 8288 mini_ranger.cc:68] Stopped Ranger
2026-06-20 07:09:12.681 UTC [20093] LOG: received smart shutdown request
2026-06-20 07:09:12.684 UTC [20093] LOG: background worker "logical replication launcher" (PID 20102) exited with exit code 1
2026-06-20 07:09:12.685 UTC [20097] LOG: shutting down
2026-06-20 07:09:12.814 UTC [20093] LOG: database system is shut down
2026-06-20T07:09:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44853 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:09:12 dist-test-slave-wjw3 krb5kdc[21603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:09:12 dist-test-slave-wjw3 krb5kdc[21603](info): set up 2 sockets
Jun 20 07:09:12 dist-test-slave-wjw3 krb5kdc[21603](info): commencing operation
krb5kdc: starting...
W20260620 07:09:13.871327 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:09:13 dist-test-slave-wjw3 krb5kdc[21603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:09:13.981937 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35749
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:14.087302 21616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:14.087613 21616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:14.087677 21616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:14.091670 21616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:14.091733 21616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:14.091759 21616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:14.091779 21616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:14.091809 21616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:14.097076 21616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:14.098101 21616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:14.099086 21616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:14.107012 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:14.107010 21624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:14.107005 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:14.107376 21616 server_base.cc:1034] running on GCE node
I20260620 07:09:14.107924 21616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:14.108963 21616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:14.110114 21616 hybrid_clock.cc:648] HybridClock initialized: now 1781939354110095 us; error 32 us; skew 500 ppm
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:14.112658 21616 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:14.113519 21616 webserver.cc:460] Webserver started at http://127.8.24.62:36385/ using document root <none> and password file <none>
I20260620 07:09:14.114024 21616 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:14.114087 21616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:14.114289 21616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:14.116293 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "9111e4d1991c4658ade62a4531a293cb"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "cd380ad88dd4b9becd697244d8d8a76f"
server_key_iv: "58e04adcb4e8ed86ce80feadcb5e5bf9"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.116816 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "9111e4d1991c4658ade62a4531a293cb"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "cd380ad88dd4b9becd697244d8d8a76f"
server_key_iv: "58e04adcb4e8ed86ce80feadcb5e5bf9"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.120273 21616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:14.122510 21631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:14.123684 21616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:14.123790 21616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "9111e4d1991c4658ade62a4531a293cb"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "cd380ad88dd4b9becd697244d8d8a76f"
server_key_iv: "58e04adcb4e8ed86ce80feadcb5e5bf9"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.123876 21616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:14.138721 21616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:14.141377 21616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:14.141544 21616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:14.148268 21616 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35749
I20260620 07:09:14.148293 21683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35749 every 8 connection(s)
I20260620 07:09:14.149163 21616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:14.151860 21684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:14.155427 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.156877 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21616
I20260620 07:09:14.156991 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:09:14.157260 8288 external_mini_cluster.cc:1428] Setting key e71220f2a7fe9394e743586ef2f28d45
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:14.172008 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.157649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35749 (local address 127.0.0.1:45566)
0620 07:09:14.158100 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.158151 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.158979 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.159673 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.159678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.160087 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.160320 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.160332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.161493 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.161496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.162208 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.162217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.162802 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.162805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.162919 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.163449 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.163471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.167162 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.169077 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.169086 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.169097 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.169380 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.169711 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.169715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.169717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.169859 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.170469 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.170477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.170644 (+ 167us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.170840 (+ 196us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.171224 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":86,"thread_start_us":99,"threads_started":1}
I20260620 07:09:14.172008 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.157651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.8.24.62:35749)
0620 07:09:14.158161 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.158172 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.158249 (+ 77us) server_negotiation.cc:407] Connection header received
0620 07:09:14.159275 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.159279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.159496 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.159657 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.160458 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.161383 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.162318 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.162666 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.162805 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.167320 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.167326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.167341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.167367 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.168980 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.169510 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.169515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.169516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.169622 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.169991 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.169994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.169995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.170355 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.170456 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.170882 (+ 426us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.171130 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":405,"thread_start_us":134,"threads_started":1}
I20260620 07:09:14.174505 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.175184 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.174926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39796)
0620 07:09:14.175027 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.175114 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 07:09:14.175357 21632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
I20260620 07:09:14.176177 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.175990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39806)
0620 07:09:14.176077 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.176127 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 07:09:14.176291 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40431: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260620 07:09:14.180454 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40431
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:14.225546 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } attempt: 1
I20260620 07:09:14.226468 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.226120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39816)
0620 07:09:14.226279 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.226353 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260620 07:09:14.227173 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.226950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39826)
0620 07:09:14.227048 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.227099 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 07:09:14.227308 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40431: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
W20260620 07:09:14.287074 21691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:14.287338 21691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:14.287396 21691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:14.291337 21691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:14.291401 21691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:14.291428 21691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:14.291471 21691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:14.291492 21691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:14.295475 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } attempt: 2
I20260620 07:09:14.296283 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.295965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39842)
0620 07:09:14.296130 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.296198 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 07:09:14.296921 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.296720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39852)
0620 07:09:14.296810 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.296860 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 07:09:14.297091 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40431: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40431: connect: Connection refused (error 111)
I20260620 07:09:14.297080 21691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:14.298089 21691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:14.299048 21691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:14.306811 21697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:14.306825 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:14.306821 21699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:14.306998 21691 server_base.cc:1034] running on GCE node
I20260620 07:09:14.307654 21691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:14.308684 21691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:14.309831 21691 hybrid_clock.cc:648] HybridClock initialized: now 1781939354309805 us; error 38 us; skew 500 ppm
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:14.312379 21691 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260620 07:09:14.313216 21691 webserver.cc:460] Webserver started at http://127.8.24.61:38835/ using document root <none> and password file <none>
I20260620 07:09:14.313767 21691 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:14.313835 21691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:14.314028 21691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:14.316067 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data/instance:
uuid: "ba7299836a3448cca1a6ae6a70871951"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "244c58736b2e2196043d51066b61259d"
server_key_iv: "657fe316419a8121f36811d5d7f3cc21"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.316604 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal/instance:
uuid: "ba7299836a3448cca1a6ae6a70871951"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "244c58736b2e2196043d51066b61259d"
server_key_iv: "657fe316419a8121f36811d5d7f3cc21"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.320206 21691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 07:09:14.322532 21706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:14.323724 21691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:14.323837 21691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "ba7299836a3448cca1a6ae6a70871951"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "244c58736b2e2196043d51066b61259d"
server_key_iv: "657fe316419a8121f36811d5d7f3cc21"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.323927 21691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:14.352109 21691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:14.355149 21691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:14.355315 21691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:14.361753 21691 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:40431
I20260620 07:09:14.361783 21758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:40431 every 8 connection(s)
I20260620 07:09:14.362627 21691 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
I20260620 07:09:14.365154 21759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:14.366005 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21691
I20260620 07:09:14.366148 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-1/wal/instance
I20260620 07:09:14.366434 8288 external_mini_cluster.cc:1428] Setting key 0e66725941040bbc2e177b2c414b0fb7
I20260620 07:09:14.369387 21759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:14.382628 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.371594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.8.24.62:35749)
0620 07:09:14.371753 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.371759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.372530 (+ 771us) server_negotiation.cc:407] Connection header received
0620 07:09:14.373467 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.373471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.373530 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.373640 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.374768 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.375308 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.376613 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.376884 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.376995 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.379240 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.379243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.379245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.379273 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.380651 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.381140 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.381143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.381144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.381221 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.381526 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.381529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.381531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.381716 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.381801 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.382326 (+ 525us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.382458 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 07:09:14.383463 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.371908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35749 (local address 127.0.0.1:45580)
0620 07:09:14.372373 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.372424 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.373254 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.373777 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.373783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.374101 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.374628 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.374643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.375413 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.375415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.376481 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.376494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.377027 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.377030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.377180 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.377732 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.377751 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.379074 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.380765 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.380771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.380788 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.381047 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.381301 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.381304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.381306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.381442 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.381814 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.381821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.382086 (+ 265us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.382254 (+ 168us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.382592 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
I20260620 07:09:14.384845 21759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.388597 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } attempt: 3
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260620 07:09:14.391957 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:37605
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:14.394419 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.385309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39868)
0620 07:09:14.385429 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.385443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.385523 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.386017 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.386019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.386143 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.386228 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.386233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.387289 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.387293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.387891 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.387900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.388689 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.388693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.388806 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.389249 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.389263 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.390751 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.392805 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.392807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.392809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.393007 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.393326 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.393328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.393330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.393411 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.393928 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.393931 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.394026 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.394132 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.394257 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":16}
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:09:14.394912 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.385402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.8.24.61:40431)
0620 07:09:14.385678 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.385687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.385709 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:09:14.385751 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.385757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.385912 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.386012 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.386332 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.387179 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.387994 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.388523 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.388691 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.390884 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.390894 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.390908 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.390931 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.392703 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.393129 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.393135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.393137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.393226 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.393521 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.393525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.393526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.393798 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.393932 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.394239 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.394383 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260620 07:09:14.396775 21759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.397541 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.397274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40088)
0620 07:09:14.397393 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.397470 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 07:09:14.397722 21708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
I20260620 07:09:14.398586 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.398370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40100)
0620 07:09:14.398476 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.398525 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 07:09:14.398684 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.389015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39882)
0620 07:09:14.389146 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.389161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.389239 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.389986 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.389989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.390229 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.390770 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.390779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.391931 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.391933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.392455 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.392463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.392929 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.392932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.393034 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.393465 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.393480 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.394808 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.397206 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.397209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.397211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.397432 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.397733 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.397736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.397738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.397828 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.398239 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.398242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.398339 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.398406 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.398532 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 07:09:14.398718 21759 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
I20260620 07:09:14.399173 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.389454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.8.24.61:40431)
0620 07:09:14.389748 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.389757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.389771 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:09:14.389803 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.389806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.389845 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.389926 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.390870 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.391660 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.392562 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.392831 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.392919 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.394952 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.394955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.394958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.394982 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.397112 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.397549 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.397553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.397555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.397631 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.397938 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.397941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.397943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.398132 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.398853 (+ 721us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.398950 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.399059 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
I20260620 07:09:14.399791 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.400405 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.400201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40106)
0620 07:09:14.400304 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.400352 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 07:09:14.401065 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.400822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40122)
0620 07:09:14.400965 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.401018 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260620 07:09:14.401208 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
I20260620 07:09:14.432384 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 1
I20260620 07:09:14.433195 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.432909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40136)
0620 07:09:14.433055 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.433117 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 07:09:14.433974 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.433763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40144)
0620 07:09:14.433861 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.433909 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 07:09:14.434135 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
I20260620 07:09:14.448064 21759 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 1
I20260620 07:09:14.448868 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.448603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40150)
0620 07:09:14.448734 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.448792 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 07:09:14.449795 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.449587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40158)
0620 07:09:14.449693 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.449739 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 07:09:14.449954 21759 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
W20260620 07:09:14.504503 21766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:14.504762 21766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:14.504803 21766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:14.508778 21766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:14.508844 21766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:14.508862 21766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:14.508877 21766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:14.508893 21766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:14.509282 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 2
I20260620 07:09:14.510114 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.509812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40168)
0620 07:09:14.509960 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.510035 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 07:09:14.510782 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.510583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40176)
0620 07:09:14.510685 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.510728 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 07:09:14.510921 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
I20260620 07:09:14.514508 21766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35749,127.8.24.61:40431,127.8.24.60:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:14.515594 21766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:14.516592 21766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:09:14.518131 21759 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 2
I20260620 07:09:14.518834 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.518579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40192)
0620 07:09:14.518710 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.518763 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 07:09:14.519475 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.519248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40208)
0620 07:09:14.519345 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.519388 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 07:09:14.519636 21759 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:37605: Network error: Client connection negotiation failed: client connection to 127.8.24.60:37605: connect: Connection refused (error 111)
W20260620 07:09:14.524885 21772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:14.524983 21766 server_base.cc:1034] running on GCE node
W20260620 07:09:14.524891 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:14.524885 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:14.525671 21766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:14.526751 21766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:14.527902 21766 hybrid_clock.cc:648] HybridClock initialized: now 1781939354527873 us; error 41 us; skew 500 ppm
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:14.530570 21766 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260620 07:09:14.531427 21766 webserver.cc:460] Webserver started at http://127.8.24.60:45593/ using document root <none> and password file <none>
I20260620 07:09:14.531980 21766 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:14.532042 21766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:14.532235 21766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:14.534264 21766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data/instance:
uuid: "03af37d4f24c48a2ab24b01e9fa5bade"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "11e2355e16848532fd08088ad2d94103"
server_key_iv: "cb78daa39122d7124f1dcc3aef646c89"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.534814 21766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal/instance:
uuid: "03af37d4f24c48a2ab24b01e9fa5bade"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "11e2355e16848532fd08088ad2d94103"
server_key_iv: "cb78daa39122d7124f1dcc3aef646c89"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.538412 21766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:09:14.540858 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:14.542073 21766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:09:14.542178 21766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "03af37d4f24c48a2ab24b01e9fa5bade"
format_stamp: "Formatted at 2026-06-20 07:09:14 on dist-test-slave-wjw3"
server_key: "11e2355e16848532fd08088ad2d94103"
server_key_iv: "cb78daa39122d7124f1dcc3aef646c89"
server_key_version: "encryptionkey@0"
I20260620 07:09:14.542286 21766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:14.560814 21766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:14.563877 21766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:14.564052 21766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:14.570971 21766 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:37605
I20260620 07:09:14.571004 21834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:37605 every 8 connection(s)
I20260620 07:09:14.572001 21766 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
I20260620 07:09:14.574779 21835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:14.578585 21835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:09:14.578879 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21766
I20260620 07:09:14.578989 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781939200335300-8288-0/minicluster-data/master-2/wal/instance
I20260620 07:09:14.579228 8288 external_mini_cluster.cc:1428] Setting key 3bc81f743caeaf18d72222a0f8f36b29
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:09:14.591996 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.581118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.8.24.62:35749)
0620 07:09:14.581245 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.581250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.581472 (+ 222us) server_negotiation.cc:407] Connection header received
0620 07:09:14.581869 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.581873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.581920 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.582009 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.583417 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.584183 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.585498 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.585801 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.585887 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.588439 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.588444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.588446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.588486 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.589834 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.590344 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.590346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.590347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.590424 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.590767 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.590771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.590773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.591005 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.591103 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.591695 (+ 592us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.591823 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 07:09:14.593019 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.581005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35749 (local address 127.0.0.1:45586)
0620 07:09:14.581332 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.581380 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.581674 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.582790 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.582795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.583113 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.583277 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.583287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.584293 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.584296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.585393 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.585402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.585936 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.585940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.586052 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.586566 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.586585 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.588240 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.589936 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.589943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.589955 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.590249 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.590526 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.590529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.590531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.590683 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.591100 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.591107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.591429 (+ 322us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.591632 (+ 203us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.591977 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260620 07:09:14.593034 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.580418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.8.24.60:37605)
0620 07:09:14.580815 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.580829 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.580862 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:09:14.581628 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.581649 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.581867 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.582054 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.582765 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.584044 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.585174 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.585706 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.585859 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.588802 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.588809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.588823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.588846 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.590451 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.590834 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.590839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.590840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.590903 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.591162 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.591164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.591166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.591454 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.591641 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.591842 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.592024 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260620 07:09:14.594820 21835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:14.603377 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.595358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.8.24.61:40431)
0620 07:09:14.595493 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.595497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.595509 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:09:14.595632 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.595634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.595674 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.595739 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.596164 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.596885 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.597947 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.598263 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.598354 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.600399 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.600403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.600406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.600433 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.601867 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.602233 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.602236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.602239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.602315 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.602580 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.602583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.602585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.602789 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.602884 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.603124 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.603222 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:09:14.603377 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.595267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40431 (local address 127.0.0.1:39892)
0620 07:09:14.595396 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.595406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.595541 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.595767 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.595770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.595932 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.596063 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.596070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.596999 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.597002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.597833 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.597843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.598366 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.598369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.598462 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.599002 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.599015 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.600271 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.601964 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.601967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.601969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.602150 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.602414 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.602417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.602419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.602498 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.602881 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.602883 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.602970 (+ 87us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.603075 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.603218 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260620 07:09:14.603941 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.595257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.8.24.62:35749)
0620 07:09:14.595365 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.595369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.595461 (+ 92us) server_negotiation.cc:407] Connection header received
0620 07:09:14.595608 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.595611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.595656 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.595731 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.596367 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.597067 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.598062 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.598356 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.598442 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.600900 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.600903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.600904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.600926 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.602293 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.602669 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.602672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.602673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.602743 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.602988 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.602990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.602992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.603192 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.603426 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.603713 (+ 287us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.603811 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:09:14.604489 21835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:09:14.610061 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 3
I20260620 07:09:14.610769 21759 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } attempt: 3
I20260620 07:09:14.612321 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.604954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40228)
0620 07:09:14.605061 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.605072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.605152 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.605417 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.605420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.605539 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.605614 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.605618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.606654 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.606657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.607139 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.607146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.607628 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.607632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.607703 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.608094 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.608108 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.609234 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.610899 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.610901 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.610904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.611104 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.611366 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.611368 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.611370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.611495 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.611886 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.611889 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.611965 (+ 76us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.612055 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.612165 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 07:09:14.612741 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.605061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.8.24.60:37605)
0620 07:09:14.605145 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.605149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.605161 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:09:14.605226 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.605229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.605265 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.605327 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.605711 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.606572 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.607236 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.607526 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.607608 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.609360 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.609364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.609367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.609391 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.610794 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.611198 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.611201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.611203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.611284 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.611579 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.611582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.611584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.611774 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.611857 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.612507 (+ 650us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.612607 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 07:09:14.614042 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.605341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.8.24.61:40431)
0620 07:09:14.605440 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.605444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.605465 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:09:14.605499 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.605502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.605538 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.605598 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.606133 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.606626 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.607426 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.607708 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.607795 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.609843 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.609847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.609848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.609870 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.612247 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.612802 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.612806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.612808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.612888 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.613170 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.613173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.613174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.613365 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.613565 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.613709 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.613807 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:09:14.617313 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:09:14.617364 21835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: Bootstrap starting.
I20260620 07:09:14.619709 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.610488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40236)
0620 07:09:14.610624 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.610640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.610718 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.611191 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.611194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.611371 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.611515 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.611523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.612545 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.612549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.613352 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.613362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.614458 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.614462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.614542 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.614979 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.614992 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.616614 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.618231 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.618234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.618236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.618415 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.618688 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.618692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.618694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.618780 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.619205 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.619208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.619305 (+ 97us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.619399 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.619554 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 07:09:14.619781 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.610531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.8.24.60:37605)
0620 07:09:14.610962 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.610971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.610983 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:09:14.611015 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.611018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.611057 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.611626 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.611710 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.612422 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.613998 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.614340 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.614751 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.616728 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.616732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.616734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.616759 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.618132 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.618499 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.618502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.618504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.618584 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.618875 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.618878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.618880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.619115 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.619203 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.619545 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.619642 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":103,"threads_started":1}
I20260620 07:09:14.620617 21835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:14.621495 21835 log.cc:824] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:14.623109 21684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: Bootstrap starting.
I20260620 07:09:14.623692 21835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: No bootstrap required, opened a new log
Jun 20 07:09:14 dist-test-slave-wjw3 krb5kdc[21603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939354, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:09:14.625771 21684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:14.626796 21684 log.cc:824] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:14.627971 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.611332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:37605 (local address 127.0.0.1:40238)
0620 07:09:14.611464 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:14.611477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:14.611545 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:14.612162 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:14.612165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:14.612338 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:14.612441 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.612447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.613725 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.613728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.614486 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:14.614495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.622435 (+ 7940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.622440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:14.622535 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.622967 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:14.622983 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:14.624331 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:14.626204 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.626207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.626210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.626445 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.626843 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:14.626846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:14.626848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:14.626941 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:14.627386 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:14.627389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:14.627530 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 07:09:14.627663 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 07:09:14.627802 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 07:09:14.628110 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.611664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.8.24.60:37605)
0620 07:09:14.611929 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.611937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.611948 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:09:14.611975 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.611978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.612026 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.612751 (+ 725us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.612865 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.613610 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.621962 (+ 8352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.622301 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.622399 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.624475 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.624479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.624482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.624503 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.626085 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.626601 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.626604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.626606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.626692 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.627052 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.627056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.627057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.627263 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.627387 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.627781 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.627939 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":109,"threads_started":1}
I20260620 07:09:14.628469 21835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.628736 21835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:14.628810 21835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03af37d4f24c48a2ab24b01e9fa5bade, State: Initialized, Role: FOLLOWER
I20260620 07:09:14.629288 21835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.629750 21851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.629940 21851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.630726 21835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:14.631389 21759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: Bootstrap starting.
I20260620 07:09:14.632266 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.619523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.8.24.60:37605)
0620 07:09:14.619908 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.619912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.622771 (+ 2859us) server_negotiation.cc:407] Connection header received
0620 07:09:14.622804 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.622807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.622845 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.622903 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.623808 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.624518 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.625589 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.625909 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.626006 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.628684 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.628688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.628690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.628714 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.630144 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.630681 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.630685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.630686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.630766 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.631039 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.631042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.631044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.631243 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.631384 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.632023 (+ 639us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.632133 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260620 07:09:14.634150 21759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:14.634965 21759 log.cc:824] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: Log is configured to *not* fsync() on all Append() calls
W20260620 07:09:14.635613 21862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:09:14.635800 21862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:09:14.637666 21759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: No bootstrap required, opened a new log
I20260620 07:09:14.634799 21766 master_runner.cc:186] Error getting master registration for 127.8.24.62:35749: OK, instance_id { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" instance_seqno: 1781939354146395 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:09:14.638957 21766 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 07:09:14.640262 21684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: No bootstrap required, opened a new log
I20260620 07:09:14.641865 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.620327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39916 (local address 127.8.24.61:40431)
0620 07:09:14.620430 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.620435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.632790 (+ 12355us) server_negotiation.cc:407] Connection header received
0620 07:09:14.632960 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.632963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.633007 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.633051 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.633846 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.634547 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.635625 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.635913 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.635980 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.637579 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.637583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.637586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.637623 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.639199 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.640496 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.640500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.640501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.640590 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.640896 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.640900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.640902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.641110 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.641220 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.641540 (+ 320us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.641666 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:09:14.641767 21759 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.642469 21759 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:14.642529 21759 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7299836a3448cca1a6ae6a70871951, State: Initialized, Role: FOLLOWER
I20260620 07:09:14.643129 21759 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.643667 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.643805 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.644517 21759 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:14.646806 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:14.619945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.8.24.62:35749)
0620 07:09:14.620044 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:14.620048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:14.626293 (+ 6245us) server_negotiation.cc:407] Connection header received
0620 07:09:14.626450 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:14.626453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:14.626520 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:14.626572 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:14.635806 (+ 9234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.636510 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.637580 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:14.637887 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:14.637957 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:14.643556 (+ 5599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:14.643560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:14.643563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:14.643590 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:14.645145 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.645610 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.645614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.645616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.645700 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:14.645976 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:14.645980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:14.645981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:14.646175 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:14.646291 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:14.646547 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 07:09:14.646660 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 07:09:14.647676 21684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.647886 21684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:14.647972 21684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9111e4d1991c4658ade62a4531a293cb, State: Initialized, Role: FOLLOWER
I20260620 07:09:14.648401 21684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.648825 21877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.649024 21877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.649688 21684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: configured and running, proceeding with master startup.
W20260620 07:09:14.655205 21888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:09:14.655330 21888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:09:14.652694 21691 master_runner.cc:186] Error getting master registration for 127.8.24.62:35749: OK, instance_id { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" instance_seqno: 1781939354146395 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:09:14.656059 21691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 07:09:14.658452 21616 master_runner.cc:263] No leader master found from master 9111e4d1991c4658ade62a4531a293cb
I20260620 07:09:14.658509 21616 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 07:09:14.658764 21876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:09:14.658821 21876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:09:14.789319 21851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:14.789517 21851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.790560 21851 leader_election.cc:288] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9111e4d1991c4658ade62a4531a293cb (127.8.24.62:35749), ba7299836a3448cca1a6ae6a70871951 (127.8.24.61:40431)
I20260620 07:09:14.791354 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9111e4d1991c4658ade62a4531a293cb" is_pre_election: true
I20260620 07:09:14.791697 21658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03af37d4f24c48a2ab24b01e9fa5bade in term 0.
I20260620 07:09:14.792240 21785 leader_election.cc:302] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03af37d4f24c48a2ab24b01e9fa5bade, 9111e4d1991c4658ade62a4531a293cb; no voters:
I20260620 07:09:14.792598 21851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:14.792672 21851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:14.792708 21851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:14.793983 21851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.795331 21851 leader_election.cc:288] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [CANDIDATE]: Term 1 election: Requested vote from peers 9111e4d1991c4658ade62a4531a293cb (127.8.24.62:35749), ba7299836a3448cca1a6ae6a70871951 (127.8.24.61:40431)
I20260620 07:09:14.795746 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9111e4d1991c4658ade62a4531a293cb"
I20260620 07:09:14.795866 21658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:14.797077 21658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03af37d4f24c48a2ab24b01e9fa5bade in term 1.
I20260620 07:09:14.797988 21733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7299836a3448cca1a6ae6a70871951" is_pre_election: true
I20260620 07:09:14.798300 21733 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03af37d4f24c48a2ab24b01e9fa5bade in term 0.
I20260620 07:09:14.799165 21733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7299836a3448cca1a6ae6a70871951"
I20260620 07:09:14.799279 21733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:14.800570 21733 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03af37d4f24c48a2ab24b01e9fa5bade in term 1.
I20260620 07:09:14.797477 21785 leader_election.cc:302] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03af37d4f24c48a2ab24b01e9fa5bade, 9111e4d1991c4658ade62a4531a293cb; no voters:
I20260620 07:09:14.801043 21851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:14.801293 21851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [term 1 LEADER]: Becoming Leader. State: Replica: 03af37d4f24c48a2ab24b01e9fa5bade, State: Running, Role: LEADER
I20260620 07:09:14.801769 21851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } }
I20260620 07:09:14.803653 21851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03af37d4f24c48a2ab24b01e9fa5bade. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.804761 21851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:14.805542 21893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:14.806507 21893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:14.810767 21733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [term 1 FOLLOWER]: Refusing update from remote peer 03af37d4f24c48a2ab24b01e9fa5bade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:14.810791 21658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [term 1 FOLLOWER]: Refusing update from remote peer 03af37d4f24c48a2ab24b01e9fa5bade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:14.811393 21851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [LEADER]: Connected to new peer: Peer: permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:14.811604 21891 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:14.815162 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03af37d4f24c48a2ab24b01e9fa5bade. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.815162 21877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03af37d4f24c48a2ab24b01e9fa5bade. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.815284 21877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.815284 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 07:09:14.815481 21788 tablet.cc:2369] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:14.818427 21851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.818540 21851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:14.818773 21894 mvcc.cc:205] Tried to move back new op lower bound from 7298823597298970624 to 7298823597271425024. Current Snapshot: MvccSnapshot[applied={T|T < 7298823597298970624}]
I20260620 07:09:14.819721 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.819828 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.821110 21891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.821215 21891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:14.822520 21877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9111e4d1991c4658ade62a4531a293cb" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35749 } } peers { permanent_uuid: "ba7299836a3448cca1a6ae6a70871951" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40431 } } peers { permanent_uuid: "03af37d4f24c48a2ab24b01e9fa5bade" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 37605 } } }
I20260620 07:09:14.822646 21877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:09:14.822957 21896 mvcc.cc:205] Tried to move back new op lower bound from 7298823597298970624 to 7298823597271425024. Current Snapshot: MvccSnapshot[applied={T|T < 7298823597298970624}]
I20260620 07:09:14.823288 21895 mvcc.cc:205] Tried to move back new op lower bound from 7298823597298970624 to 7298823597271425024. Current Snapshot: MvccSnapshot[applied={T|T < 7298823597298970624}]
I20260620 07:09:14.824862 21893 catalog_manager.cc:1347] Generated new cluster ID: 29e25cf37fb34e70995fbb8e385b84bf
I20260620 07:09:14.824918 21893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:14.835026 21893 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:09:14.835961 21893 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:14.848732 21893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03af37d4f24c48a2ab24b01e9fa5bade: Generated new TSK 0
I20260620 07:09:14.849493 21893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:09:15.656544 21888 catalog_manager.cc:1259] Loaded cluster ID: 29e25cf37fb34e70995fbb8e385b84bf
I20260620 07:09:15.656635 21888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: loading cluster ID for follower catalog manager: success
I20260620 07:09:15.658282 21888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: acquiring CA information for follower catalog manager: success
I20260620 07:09:15.659518 21888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9111e4d1991c4658ade62a4531a293cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:09:15.659950 21876 catalog_manager.cc:1259] Loaded cluster ID: 29e25cf37fb34e70995fbb8e385b84bf
I20260620 07:09:15.660012 21876 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: loading cluster ID for follower catalog manager: success
I20260620 07:09:15.661495 21876 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: acquiring CA information for follower catalog manager: success
I20260620 07:09:15.662464 21876 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba7299836a3448cca1a6ae6a70871951: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:09:15.910631 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21616
I20260620 07:09:15.915884 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21691
I20260620 07:09:15.920614 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21766
2026-06-20T07:09:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3009 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:09:15 dist-test-slave-wjw3 krb5kdc[21909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:09:15 dist-test-slave-wjw3 krb5kdc[21909](info): set up 2 sockets
Jun 20 07:09:15 dist-test-slave-wjw3 krb5kdc[21909](info): commencing operation
krb5kdc: starting...
W20260620 07:09:16.883363 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:09:16 dist-test-slave-wjw3 krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:09:16.993708 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34615
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:17.099009 21922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:17.099260 21922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:17.099315 21922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:17.103228 21922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:17.103284 21922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:17.103310 21922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:17.103332 21922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:17.103360 21922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:17.108500 21922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:17.109473 21922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:17.110415 21922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:09:17.118387 21922 server_base.cc:1034] running on GCE node
W20260620 07:09:17.118381 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:17.118363 21928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:17.118351 21927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:17.119127 21922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:17.120191 21922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:17.121415 21922 hybrid_clock.cc:648] HybridClock initialized: now 1781939357121389 us; error 37 us; skew 500 ppm
Jun 20 07:09:17 dist-test-slave-wjw3 krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939357, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:17.127048 21922 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:17.127955 21922 webserver.cc:460] Webserver started at http://127.8.24.62:46167/ using document root <none> and password file <none>
I20260620 07:09:17.128489 21922 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:17.128549 21922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:17.128741 21922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:17.130707 21922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "f0148ff359bc49c7bfb984035c1e7d49"
format_stamp: "Formatted at 2026-06-20 07:09:17 on dist-test-slave-wjw3"
server_key: "a3812b343aa6bd51725eff02b55c3cb4"
server_key_iv: "1550312ff73dc9dab550551e5bb4231e"
server_key_version: "encryptionkey@0"
I20260620 07:09:17.131232 21922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "f0148ff359bc49c7bfb984035c1e7d49"
format_stamp: "Formatted at 2026-06-20 07:09:17 on dist-test-slave-wjw3"
server_key: "a3812b343aa6bd51725eff02b55c3cb4"
server_key_iv: "1550312ff73dc9dab550551e5bb4231e"
server_key_version: "encryptionkey@0"
I20260620 07:09:17.134744 21922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:17.136941 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:17.138069 21922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:09:17.138183 21922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "f0148ff359bc49c7bfb984035c1e7d49"
format_stamp: "Formatted at 2026-06-20 07:09:17 on dist-test-slave-wjw3"
server_key: "a3812b343aa6bd51725eff02b55c3cb4"
server_key_iv: "1550312ff73dc9dab550551e5bb4231e"
server_key_version: "encryptionkey@0"
I20260620 07:09:17.138274 21922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:17.183562 21922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:17.186157 21922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:17.186318 21922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:17.192557 21922 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:34615
I20260620 07:09:17.192574 21989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:34615 every 8 connection(s)
I20260620 07:09:17.193445 21922 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:17.196035 21990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:17.199261 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 21922
I20260620 07:09:17.199376 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:09:17.199694 8288 external_mini_cluster.cc:1428] Setting key 89ab011e108c977b5874d5289f76169e
I20260620 07:09:17.201697 21990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: Bootstrap starting.
I20260620 07:09:17.204118 21990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:17.204869 21990 log.cc:824] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:17.206804 21990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: No bootstrap required, opened a new log
Jun 20 07:09:17 dist-test-slave-wjw3 krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:17.210004 21990 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } }
I20260620 07:09:17.210232 21990 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:17.210289 21990 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0148ff359bc49c7bfb984035c1e7d49, State: Initialized, Role: FOLLOWER
I20260620 07:09:17.210695 21990 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } }
I20260620 07:09:17.210827 21990 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:17.210891 21990 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:17.210981 21990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:17.211804 21990 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } }
I20260620 07:09:17.212112 21990 leader_election.cc:302] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0148ff359bc49c7bfb984035c1e7d49; no voters:
I20260620 07:09:17.212383 21990 leader_election.cc:288] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:09:17.212514 21995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:17.212707 21995 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [term 1 LEADER]: Becoming Leader. State: Replica: f0148ff359bc49c7bfb984035c1e7d49, State: Running, Role: LEADER
I20260620 07:09:17.213054 21995 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } }
I20260620 07:09:17.213462 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:17.200912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.8.24.62:34615)
0620 07:09:17.201296 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:17.201312 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:17.201346 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:09:17.202002 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:17.202020 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:17.202364 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:17.202649 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:17.203525 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:17.204513 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:17.205552 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:17.205990 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:17.206099 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:17.208618 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:17.208623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:17.208634 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:17.208657 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:17.210668 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:17.211078 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:17.211083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:17.211087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:17.211188 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:17.211426 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:17.211429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:17.211454 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:17.211816 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:17.211983 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:17.212235 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 07:09:17.212515 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":120,"threads_started":1}
I20260620 07:09:17.213564 21990 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:17.215302 21997 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0148ff359bc49c7bfb984035c1e7d49. Latest consensus state: current_term: 1 leader_uuid: "f0148ff359bc49c7bfb984035c1e7d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } } }
I20260620 07:09:17.215423 21997 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:17.215574 21996 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0148ff359bc49c7bfb984035c1e7d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0148ff359bc49c7bfb984035c1e7d49" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34615 } } }
I20260620 07:09:17.216228 21996 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:17.216538 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260620 07:09:17.217922 22009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:09:17.217981 22009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:09:17.218046 22010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:17.218713 22010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:17.223726 22010 catalog_manager.cc:1347] Generated new cluster ID: 42d380dd60e848c6b433f435595e2b98
I20260620 07:09:17.223784 22010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:17.241232 22010 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:09:17.242475 22010 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:17.253206 22010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0148ff359bc49c7bfb984035c1e7d49: Generated new TSK 0
I20260620 07:09:17.253904 22010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:09:17.271701 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 21922
2026-06-20T07:09:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1350 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:09:17 dist-test-slave-wjw3 krb5kdc[22017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:09:17 dist-test-slave-wjw3 krb5kdc[22017](info): set up 2 sockets
Jun 20 07:09:17 dist-test-slave-wjw3 krb5kdc[22017](info): commencing operation
krb5kdc: starting...
W20260620 07:09:18.223384 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:09:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:09:18.334794 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:18.441061 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:18.441289 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:18.441332 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:18.445206 22030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:18.445257 22030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:18.445277 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:18.445307 22030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:18.445333 22030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:18.450480 22030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:18.451467 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:18.452422 22030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:18.460171 22038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:18.460192 22035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:18.460171 22036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:18.460376 22030 server_base.cc:1034] running on GCE node
I20260620 07:09:18.460919 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:18.461910 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:18.463055 22030 hybrid_clock.cc:648] HybridClock initialized: now 1781939358463041 us; error 28 us; skew 500 ppm
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:18.465529 22030 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:18.466305 22030 webserver.cc:460] Webserver started at http://127.8.24.62:44787/ using document root <none> and password file <none>
I20260620 07:09:18.466799 22030 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:18.466859 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:18.467046 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:18.469024 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "8b1e5632bf2b46e08e264332aa2bfbf2"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "06267e8e25b61dde880df18223aef343"
server_key_iv: "56c60a00d227440bdb1dfcdfab6b03f0"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.469571 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "8b1e5632bf2b46e08e264332aa2bfbf2"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "06267e8e25b61dde880df18223aef343"
server_key_iv: "56c60a00d227440bdb1dfcdfab6b03f0"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.473953 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 07:09:18.476253 22045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.477385 22030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:18.477501 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "8b1e5632bf2b46e08e264332aa2bfbf2"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "06267e8e25b61dde880df18223aef343"
server_key_iv: "56c60a00d227440bdb1dfcdfab6b03f0"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.477592 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:18.489054 22030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:18.491966 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:18.492128 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:18.498490 22030 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44247
I20260620 07:09:18.498720 22097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44247 every 8 connection(s)
I20260620 07:09:18.499354 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:18.500329 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22030
I20260620 07:09:18.500458 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:09:18.500764 8288 external_mini_cluster.cc:1428] Setting key 2c0c54a40f9c37f4a227dba80984d969
I20260620 07:09:18.502466 22098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:18.508606 22098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap starting.
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:18.510782 22098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:18.511482 22098 log.cc:824] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:18.513302 22098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: No bootstrap required, opened a new log
I20260620 07:09:18.514024 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:18.502048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.8.24.62:44247)
0620 07:09:18.502408 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:18.502429 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:18.502464 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:09:18.503112 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:18.503128 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:18.503482 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:18.503756 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:18.504557 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.505532 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.506630 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.507081 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.507188 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:18.509782 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:18.509788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:18.509800 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:18.509826 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:18.511392 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:18.511841 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:18.511846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:18.511850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:18.511959 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:18.512205 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:18.512209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:18.512210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:18.512587 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:18.512754 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:18.513020 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 07:09:18.513293 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260620 07:09:18.516996 22098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:18.517210 22098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:18.517266 22098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Initialized, Role: FOLLOWER
I20260620 07:09:18.517659 22098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:18.517783 22098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:18.517844 22098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:18.517935 22098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:18.518740 22098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:18.519048 22098 leader_election.cc:302] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b1e5632bf2b46e08e264332aa2bfbf2; no voters:
I20260620 07:09:18.519321 22098 leader_election.cc:288] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:09:18.519423 22103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:18.519646 22103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Running, Role: LEADER
I20260620 07:09:18.519970 22103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:18.520483 22098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:18.521548 22104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:18.521543 22105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b1e5632bf2b46e08e264332aa2bfbf2. Latest consensus state: current_term: 1 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:18.521692 22104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:18.521695 22105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:18.522045 22112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:18.525040 22112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:18.530366 22112 catalog_manager.cc:1347] Generated new cluster ID: dad8e39c94c44677b6006f5c32f76314
I20260620 07:09:18.530428 22112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:18.554163 22112 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:09:18.555097 22112 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:18.566321 22112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Generated new TSK 0
I20260620 07:09:18.566958 22112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:09:18.598537 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:18.705670 22124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:18.705930 22124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:18.705987 22124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:18.709913 22124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:18.709968 22124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:18.710044 22124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:18.715149 22124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:18.716184 22124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:18.717154 22124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:18.726140 22130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:18.726192 22124 server_base.cc:1034] running on GCE node
W20260620 07:09:18.726194 22129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:18.726140 22132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:18.726792 22124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:18.727396 22124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:18.728600 22124 hybrid_clock.cc:648] HybridClock initialized: now 1781939358728577 us; error 39 us; skew 500 ppm
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:18.731047 22124 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:18.731866 22124 webserver.cc:460] Webserver started at http://127.8.24.1:33119/ using document root <none> and password file <none>
I20260620 07:09:18.732371 22124 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:18.732427 22124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:18.732614 22124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:18.734541 22124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "98b3cacc0a37ae43d2a3c45195d77f96"
server_key_iv: "f2ecbda28f3f823b952be6fc5e95293f"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.735059 22124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "98b3cacc0a37ae43d2a3c45195d77f96"
server_key_iv: "f2ecbda28f3f823b952be6fc5e95293f"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.738540 22124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:09:18.740808 22139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.741925 22124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260620 07:09:18.742015 22124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "98b3cacc0a37ae43d2a3c45195d77f96"
server_key_iv: "f2ecbda28f3f823b952be6fc5e95293f"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.742133 22124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:18.761729 22124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:18.764724 22124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:18.764892 22124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:18.765453 22124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:18.766240 22124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:18.766285 22124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.766325 22124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:18.766353 22124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.774536 22124 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35621
I20260620 07:09:18.774569 22252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35621 every 8 connection(s)
I20260620 07:09:18.775383 22124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:18.784643 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22124
I20260620 07:09:18.784772 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:09:18.785032 8288 external_mini_cluster.cc:1428] Setting key b299e0e6201d8469f889ee7bbffd55bc
I20260620 07:09:18.788281 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:18.777084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34345 (local address 127.8.24.62:44247)
0620 07:09:18.777217 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:18.777220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:18.778007 (+ 787us) server_negotiation.cc:407] Connection header received
0620 07:09:18.779250 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:18.779254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:18.779311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:18.779385 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:18.780574 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.781115 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.782232 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.782480 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.782570 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:18.784866 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:18.784869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:18.784872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:18.784902 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:18.786263 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:18.786725 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:18.786730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:18.786731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:18.786822 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:18.787091 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:18.787094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:18.787095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:18.787294 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:18.787385 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:18.787989 (+ 604us) server_negotiation.cc:299] Negotiation successful
0620 07:09:18.788120 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:09:18.789067 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:18.777430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:34345)
0620 07:09:18.777863 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:18.777906 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:18.779037 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:18.779586 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:18.779595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:18.779935 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:18.780433 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:18.780446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.781220 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.781224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:18.782117 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:18.782126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.782617 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.782622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:18.782778 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:18.783263 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:18.783280 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:18.784714 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:18.786376 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:18.786381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:18.786393 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:18.786634 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:18.786898 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:18.786900 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:18.786901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:18.787020 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:18.787465 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:18.787472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:18.787721 (+ 249us) client_negotiation.cc:769] Sending connection context
0620 07:09:18.787907 (+ 186us) client_negotiation.cc:240] Negotiation successful
0620 07:09:18.788234 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260620 07:09:18.790426 22253 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:18.790656 22253 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:18.791107 22253 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:18.792582 22062 ts_manager.cc:194] Registered new tserver with Master: 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621)
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:09:18.793660 22062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:34345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:09:18.808641 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:18.915772 22258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:18.916021 22258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:18.916077 22258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:18.919992 22258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:18.920049 22258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:18.920125 22258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:18.925257 22258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:18.926225 22258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:18.927177 22258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:18.936239 22264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:18.936327 22258 server_base.cc:1034] running on GCE node
W20260620 07:09:18.936247 22263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:18.936245 22266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:18.936825 22258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:18.937412 22258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:18.938575 22258 hybrid_clock.cc:648] HybridClock initialized: now 1781939358938554 us; error 34 us; skew 500 ppm
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:18.941148 22258 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:18.941962 22258 webserver.cc:460] Webserver started at http://127.8.24.2:33733/ using document root <none> and password file <none>
I20260620 07:09:18.942456 22258 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:18.942512 22258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:18.942670 22258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:18.944602 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "d0214eb22d87398c73346d8d08237f34"
server_key_iv: "ac913e398e2782f119fd9e82e0d72413"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.945130 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "d0214eb22d87398c73346d8d08237f34"
server_key_iv: "ac913e398e2782f119fd9e82e0d72413"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.948738 22258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:18.950961 22273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.952160 22258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:18.952284 22258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "d0214eb22d87398c73346d8d08237f34"
server_key_iv: "ac913e398e2782f119fd9e82e0d72413"
server_key_version: "encryptionkey@0"
I20260620 07:09:18.952385 22258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:18.975701 22258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:18.978403 22258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:18.978576 22258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:18.979136 22258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:18.979966 22258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:18.980015 22258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.980060 22258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:18.980088 22258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:18.988551 22258 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39627
I20260620 07:09:18.988567 22386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39627 every 8 connection(s)
I20260620 07:09:18.989463 22258 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:18.994024 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22258
I20260620 07:09:18.994164 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:09:18.994459 8288 external_mini_cluster.cc:1428] Setting key fa0b649807ad13a6591e47a72209551e
Jun 20 07:09:18 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:19.003010 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:18.991153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54069 (local address 127.8.24.62:44247)
0620 07:09:18.991286 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:18.991290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:18.992149 (+ 859us) server_negotiation.cc:407] Connection header received
0620 07:09:18.993400 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:18.993404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:18.993460 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:18.993532 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:18.995033 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.995737 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.996869 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.997092 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.997177 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:18.999658 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:18.999662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:18.999664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:18.999687 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:19.000969 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.001540 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.001543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.001544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.001615 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.001952 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.001954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.001955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.002116 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:19.002184 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.002783 (+ 599us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.002884 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:09:19.004172 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:18.991587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:54069)
0620 07:09:18.992007 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:18.992049 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:18.993184 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:18.993709 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:18.993716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:18.994237 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:18.994893 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:18.994912 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.995851 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.995855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:18.996756 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:18.996765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:18.997224 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:18.997229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:18.997386 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:18.997923 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:18.997939 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:18.999510 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:19.001083 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:19.001089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:19.001107 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:19.001449 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:19.001690 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:19.001693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:19.001695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:19.001874 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:19.002204 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:19.002211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:19.002532 (+ 321us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.002765 (+ 233us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.003130 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":107,"threads_started":1}
I20260620 07:09:19.005309 22387 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:19.005561 22387 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:09:19.006091 22387 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:19.007175 22062 ts_manager.cc:194] Registered new tserver with Master: 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
I20260620 07:09:19.007799 22062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:54069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:09:19.019877 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:19.130446 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:19.130713 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:19.130769 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:19.134789 22392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:19.134851 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:19.134931 22392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:19.140133 22392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:19.141134 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:19.142108 22392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:19.151166 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:19.151204 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:19.151237 22392 server_base.cc:1034] running on GCE node
W20260620 07:09:19.151178 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:19.151791 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:19.152359 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:19.153509 22392 hybrid_clock.cc:648] HybridClock initialized: now 1781939359153474 us; error 53 us; skew 500 ppm
Jun 20 07:09:19 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:19.156121 22392 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:19.156955 22392 webserver.cc:460] Webserver started at http://127.8.24.3:34759/ using document root <none> and password file <none>
I20260620 07:09:19.157459 22392 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:19.157516 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:19.157703 22392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:19.159723 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "cae94afb87d042f1aab170f49f0609cd"
format_stamp: "Formatted at 2026-06-20 07:09:19 on dist-test-slave-wjw3"
server_key: "d2e51c652204d69c7c43a8e66de7a8e0"
server_key_iv: "24c1c02d14e508579a031895e92b92bb"
server_key_version: "encryptionkey@0"
I20260620 07:09:19.160254 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "cae94afb87d042f1aab170f49f0609cd"
format_stamp: "Formatted at 2026-06-20 07:09:19 on dist-test-slave-wjw3"
server_key: "d2e51c652204d69c7c43a8e66de7a8e0"
server_key_iv: "24c1c02d14e508579a031895e92b92bb"
server_key_version: "encryptionkey@0"
I20260620 07:09:19.163808 22392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:09:19.166162 22407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:19.167351 22392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:19.167474 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "cae94afb87d042f1aab170f49f0609cd"
format_stamp: "Formatted at 2026-06-20 07:09:19 on dist-test-slave-wjw3"
server_key: "d2e51c652204d69c7c43a8e66de7a8e0"
server_key_iv: "24c1c02d14e508579a031895e92b92bb"
server_key_version: "encryptionkey@0"
I20260620 07:09:19.167577 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:19.188040 22392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:19.191093 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:19.191259 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:19.191855 22392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:19.192625 22392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:19.192670 22392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:19.192721 22392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:19.192749 22392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:19.200902 22392 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40011
I20260620 07:09:19.200932 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40011 every 8 connection(s)
I20260620 07:09:19.201897 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:19.207091 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22392
I20260620 07:09:19.207190 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:09:19.207396 8288 external_mini_cluster.cc:1428] Setting key f8cf364f082efcb6566982cc47cd82ca
Jun 20 07:09:19 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:19.214798 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.203671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54315 (local address 127.8.24.62:44247)
0620 07:09:19.203792 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.203795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.204596 (+ 801us) server_negotiation.cc:407] Connection header received
0620 07:09:19.205749 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.205752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.205797 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.205865 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:19.207722 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.208328 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.209546 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.209768 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.209841 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.212100 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:19.212103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:19.212105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:19.212129 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:19.213237 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.213673 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.213675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.213677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.213739 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.213982 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.213983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.213984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.214133 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:19.214198 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.214594 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.214682 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 07:09:19.215613 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.204023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:54315)
0620 07:09:19.204441 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.204484 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.205571 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.205997 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.206003 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.206333 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:19.206807 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.206819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.208437 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.208440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.209450 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.209459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.209878 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.209880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.210021 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.210510 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:19.210526 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:19.211949 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:19.213331 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:19.213336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:19.213348 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:19.213603 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:19.213796 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:19.213798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:19.213800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:19.213922 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:19.214196 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:19.214200 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:19.214416 (+ 216us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.214561 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.214802 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":95,"threads_started":1}
I20260620 07:09:19.216575 22521 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:19.216771 22521 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:19.217187 22521 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:19.218099 22062 ts_manager.cc:194] Registered new tserver with Master: cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011)
I20260620 07:09:19.218678 22062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:54315
I20260620 07:09:19.220891 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:19.228870 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.222754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.8.24.62:44247)
0620 07:09:19.222861 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.222864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.223058 (+ 194us) server_negotiation.cc:407] Connection header received
0620 07:09:19.223154 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.223156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.223194 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.223247 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:19.223931 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.224422 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.225176 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.225369 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.225434 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.226602 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:19.226604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:19.226606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:19.226620 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:19.227679 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.228001 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.228004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.228005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.228064 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:19.228245 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:19.228247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:19.228248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:19.228431 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:19.228524 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.228683 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.228764 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:09:19.231208 22062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:09:19.234120 22062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:09:19.247574 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.242578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40011 (local address 127.0.0.1:36754)
0620 07:09:19.242830 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.242842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.242909 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.243599 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.243602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.243618 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:19.243688 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.243693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.245092 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.245095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.246095 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.246104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.247019 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.247140 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247178 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.247276 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.247384 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260620 07:09:19.247594 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.241719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35621 (local address 127.0.0.1:55128)
0620 07:09:19.242030 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.242074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.242187 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.243239 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.243247 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.243271 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:19.243403 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.243410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.244653 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.244656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.246155 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.246166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.247231 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.247333 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247354 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.247405 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.247488 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":34,"thread_start_us":71,"threads_started":1}
I20260620 07:09:19.247718 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.242384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55128 (local address 127.8.24.1:35621)
0620 07:09:19.242728 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.242745 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.242766 (+ 21us) server_negotiation.cc:407] Connection header received
0620 07:09:19.242842 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.242847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.243051 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.243592 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:19.243753 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.244533 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.246290 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.247100 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247278 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247393 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.247474 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.247580 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":78,"threads_started":1}
I20260620 07:09:19.248003 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.242833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.8.24.3:40011)
0620 07:09:19.243180 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.243191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.243210 (+ 19us) server_negotiation.cc:407] Connection header received
0620 07:09:19.243275 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.243279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.243461 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.243817 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:19.243965 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.244985 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.246223 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.246916 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247009 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247153 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.247534 (+ 381us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.247619 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":147,"threads_started":1}
I20260620 07:09:19.248235 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.241823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39627 (local address 127.0.0.1:35860)
0620 07:09:19.242569 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.242581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.242678 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.243356 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.243360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.243377 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:19.243495 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.243502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.244901 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.244904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.246166 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.246172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.247226 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.247713 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247985 (+ 272us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.248047 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.248113 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":92,"thread_start_us":93,"threads_started":1}
I20260620 07:09:19.248322 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.241981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.8.24.2:39627)
0620 07:09:19.242446 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.242457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.242768 (+ 311us) server_negotiation.cc:407] Connection header received
0620 07:09:19.242844 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.242850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.243047 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.243183 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:19.243619 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.244772 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.246290 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.247100 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.247200 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.247332 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.248105 (+ 773us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.248196 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":111,"threads_started":1}
I20260620 07:09:19.250545 22455 tablet_service.cc:1467] Processing CreateTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 (DEFAULT_TABLE table=test-table [id=1ab00a5c58db47d08450fcc55679aaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:19.250764 22321 tablet_service.cc:1467] Processing CreateTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 (DEFAULT_TABLE table=test-table [id=1ab00a5c58db47d08450fcc55679aaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:19.250701 22187 tablet_service.cc:1467] Processing CreateTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 (DEFAULT_TABLE table=test-table [id=1ab00a5c58db47d08450fcc55679aaed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:19.251513 22455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fbbc28ebd34c28b009fc6838df41d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:19.251693 22321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fbbc28ebd34c28b009fc6838df41d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:19.251693 22187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6fbbc28ebd34c28b009fc6838df41d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:19.258190 22538 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Bootstrap starting.
I20260620 07:09:19.258818 22540 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Bootstrap starting.
I20260620 07:09:19.259758 22539 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap starting.
I20260620 07:09:19.260421 22538 tablet_bootstrap.cc:654] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:19.261036 22540 tablet_bootstrap.cc:654] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:19.261391 22538 log.cc:824] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:19.261656 22539 tablet_bootstrap.cc:654] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:19.261833 22540 log.cc:824] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:19.262733 22539 log.cc:824] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:19.263474 22538 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: No bootstrap required, opened a new log
I20260620 07:09:19.263700 22538 ts_tablet_manager.cc:1397] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:09:19.263775 22540 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: No bootstrap required, opened a new log
I20260620 07:09:19.263944 22540 ts_tablet_manager.cc:1397] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260620 07:09:19.264541 22539 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: No bootstrap required, opened a new log
I20260620 07:09:19.264712 22539 ts_tablet_manager.cc:1397] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:09:19.266568 22538 raft_consensus.cc:348] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.266788 22538 raft_consensus.cc:374] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:19.266855 22538 raft_consensus.cc:729] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae94afb87d042f1aab170f49f0609cd, State: Initialized, Role: FOLLOWER
I20260620 07:09:19.267284 22538 consensus_queue.cc:260] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.267624 22540 raft_consensus.cc:348] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.267886 22521 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
I20260620 07:09:19.267931 22540 raft_consensus.cc:374] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:19.268067 22540 raft_consensus.cc:729] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5bf3ea8ec04bde8d5e6483e164f7fb, State: Initialized, Role: FOLLOWER
I20260620 07:09:19.268260 22538 ts_tablet_manager.cc:1428] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:09:19.268278 22539 raft_consensus.cc:348] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.268567 22539 raft_consensus.cc:374] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:19.268518 22540 consensus_queue.cc:260] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.268625 22539 raft_consensus.cc:729] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcf21e68e0e4606915a5fed7c588cc7, State: Initialized, Role: FOLLOWER
I20260620 07:09:19.269131 22387 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
I20260620 07:09:19.269228 22539 consensus_queue.cc:260] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.269440 22540 ts_tablet_manager.cc:1428] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:09:19.270013 22539 ts_tablet_manager.cc:1428] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:09:19.270340 22253 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
W20260620 07:09:19.276372 22254 tablet.cc:2369] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:09:19.452896 22522 tablet.cc:2369] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:09:19.490581 22388 tablet.cc:2369] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:19.582166 22546 raft_consensus.cc:482] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:19.582374 22546 raft_consensus.cc:504] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.583535 22546 leader_election.cc:288] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627), cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011)
I20260620 07:09:19.587217 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.583844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:53947 (local address 127.8.24.2:39627)
0620 07:09:19.584052 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.584057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.584118 (+ 61us) server_negotiation.cc:407] Connection header received
0620 07:09:19.584271 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.584274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.584324 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.584386 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:19.584658 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.585165 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586284 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.586683 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586758 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.586811 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.587021 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.587091 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260620 07:09:19.587229 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.583754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39627 (local address 127.8.24.1:53947)
0620 07:09:19.584029 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.584044 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.584167 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.584412 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.584415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.584431 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:19.584555 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.584562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.585272 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.585275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.586134 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.586145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.586791 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.586905 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.586929 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.586995 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.587064 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":4,"thread_start_us":86,"threads_started":1}
I20260620 07:09:19.587252 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.583892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:35265 (local address 127.8.24.3:40011)
0620 07:09:19.584094 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.584098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.584236 (+ 138us) server_negotiation.cc:407] Connection header received
0620 07:09:19.584357 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.584359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.584410 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.584475 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:19.584736 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.585205 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586343 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.586745 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586844 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.586903 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.587086 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.587153 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260620 07:09:19.587256 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.583792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40011 (local address 127.8.24.1:35265)
0620 07:09:19.584141 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.584154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:19.584259 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:19.584500 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:19.584503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:19.584518 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:19.584633 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.584639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.585311 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.585313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.586235 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:19.586246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.586861 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.586864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:19.586967 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.586988 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:09:19.587058 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:09:19.587126 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":109,"thread_start_us":94,"threads_started":1}
I20260620 07:09:19.587810 22341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fbbc28ebd34c28b009fc6838df41d3" candidate_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" is_pre_election: true
I20260620 07:09:19.587862 22475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fbbc28ebd34c28b009fc6838df41d3" candidate_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae94afb87d042f1aab170f49f0609cd" is_pre_election: true
I20260620 07:09:19.588119 22475 raft_consensus.cc:2454] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcf21e68e0e4606915a5fed7c588cc7 in term 0.
I20260620 07:09:19.588119 22341 raft_consensus.cc:2454] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcf21e68e0e4606915a5fed7c588cc7 in term 0.
I20260620 07:09:19.588596 22142 leader_election.cc:302] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7, 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd; no voters:
I20260620 07:09:19.588822 22546 raft_consensus.cc:2781] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:19.588896 22546 raft_consensus.cc:482] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:19.588943 22546 raft_consensus.cc:3037] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:19.589931 22546 raft_consensus.cc:504] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.590242 22546 leader_election.cc:288] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627), cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011)
I20260620 07:09:19.590550 22475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fbbc28ebd34c28b009fc6838df41d3" candidate_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cae94afb87d042f1aab170f49f0609cd"
I20260620 07:09:19.590657 22475 raft_consensus.cc:3037] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:19.591634 22341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fbbc28ebd34c28b009fc6838df41d3" candidate_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
I20260620 07:09:19.591743 22341 raft_consensus.cc:3037] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:19.591848 22475 raft_consensus.cc:2454] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcf21e68e0e4606915a5fed7c588cc7 in term 1.
I20260620 07:09:19.592110 22142 leader_election.cc:302] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7, cae94afb87d042f1aab170f49f0609cd; no voters:
I20260620 07:09:19.592263 22546 raft_consensus.cc:2781] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:19.592443 22546 raft_consensus.cc:686] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcf21e68e0e4606915a5fed7c588cc7, State: Running, Role: LEADER
I20260620 07:09:19.592779 22546 consensus_queue.cc:237] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:19.592882 22341 raft_consensus.cc:2454] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcf21e68e0e4606915a5fed7c588cc7 in term 1.
I20260620 07:09:19.595818 22060 catalog_manager.cc:5495] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } health_report { overall_health: UNKNOWN } } }
I20260620 07:09:19.608649 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.605546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.8.24.1:35621)
0620 07:09:19.605660 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:19.605664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:19.605676 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:09:19.605761 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:19.605764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:19.605816 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:19.605901 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:19.606077 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.606585 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.607359 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:19.607633 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:19.607708 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:19.607931 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:19.608329 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:19.608414 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:19.608486 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 07:09:19.608535 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:09:19.614418 22475 raft_consensus.cc:1261] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Refusing update from remote peer 1bcf21e68e0e4606915a5fed7c588cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:19.614418 22341 raft_consensus.cc:1261] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Refusing update from remote peer 1bcf21e68e0e4606915a5fed7c588cc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:19.615180 22546 consensus_queue.cc:1035] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:19.615382 22549 consensus_queue.cc:1035] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:19.622381 22554 mvcc.cc:205] Tried to move back new op lower bound from 7298823616973041664 to 7298823616894492672. Current Snapshot: MvccSnapshot[applied={T|T < 7298823616973041664}]
I20260620 07:09:19.629792 22060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:19.629961 22060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:19.632153 22060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Sending DeleteTablet for 3 replicas of tablet c6fbbc28ebd34c28b009fc6838df41d3
I20260620 07:09:19.632820 22187 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:19 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55128
I20260620 07:09:19.632925 22321 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:19 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35860
I20260620 07:09:19.633013 22455 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:19 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36754
I20260620 07:09:19.633455 22564 tablet_replica.cc:330] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: stopping tablet replica
I20260620 07:09:19.633503 22563 tablet_replica.cc:330] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: stopping tablet replica
I20260620 07:09:19.633528 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22124
I20260620 07:09:19.634366 22563 raft_consensus.cc:2229] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:19.634444 22564 raft_consensus.cc:2229] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:19.634663 22564 raft_consensus.cc:2258] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:19.634665 22563 raft_consensus.cc:2258] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:19.635974 22563 ts_tablet_manager.cc:1905] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:19.638007 22563 ts_tablet_manager.cc:1918] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:19.638098 22563 log.cc:1196] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/c6fbbc28ebd34c28b009fc6838df41d3
I20260620 07:09:19.638434 22563 ts_tablet_manager.cc:1939] T c6fbbc28ebd34c28b009fc6838df41d3 P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting consensus metadata
I20260620 07:09:19.639001 22564 ts_tablet_manager.cc:1905] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 07:09:19.639318 22049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:35621 (error 108)
I20260620 07:09:19.639592 22046 catalog_manager.cc:4841] TS 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): tablet c6fbbc28ebd34c28b009fc6838df41d3 (table test-table [id=1ab00a5c58db47d08450fcc55679aaed]) successfully deleted
I20260620 07:09:19.640187 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:19.639940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35621 (local address 127.0.0.1:55140)
0620 07:09:19.640049 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:19.640110 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 07:09:19.640319 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22258
W20260620 07:09:19.640374 22049 catalog_manager.cc:4584] TS 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6fbbc28ebd34c28b009fc6838df41d3: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35621: connect: Connection refused (error 111)
I20260620 07:09:19.641561 22564 ts_tablet_manager.cc:1918] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:19.641657 22564 log.cc:1196] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/c6fbbc28ebd34c28b009fc6838df41d3
I20260620 07:09:19.641976 22564 ts_tablet_manager.cc:1939] T c6fbbc28ebd34c28b009fc6838df41d3 P cae94afb87d042f1aab170f49f0609cd: Deleting consensus metadata
I20260620 07:09:19.642886 22048 catalog_manager.cc:4841] TS cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): tablet c6fbbc28ebd34c28b009fc6838df41d3 (table test-table [id=1ab00a5c58db47d08450fcc55679aaed]) successfully deleted
I20260620 07:09:19.645252 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22392
I20260620 07:09:19.649653 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22030
I20260620 07:09:19.654623 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--webserver_interface=127.8.24.62
--webserver_port=44787
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:19.760514 22566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:19.760766 22566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:19.760812 22566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:19.764817 22566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:19.764883 22566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:19.764907 22566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:19.764925 22566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:19.764952 22566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:19.770282 22566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=44787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:19.771373 22566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:19.772433 22566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:19.784578 22572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:19.784693 22566 server_base.cc:1034] running on GCE node
W20260620 07:09:19.784606 22573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:19.784595 22575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:19.785498 22566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:19.786623 22566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:19.787765 22566 hybrid_clock.cc:648] HybridClock initialized: now 1781939359787739 us; error 44 us; skew 500 ppm
Jun 20 07:09:19 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:19.790381 22566 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:19.791195 22566 webserver.cc:460] Webserver started at http://127.8.24.62:44787/ using document root <none> and password file <none>
I20260620 07:09:19.791862 22566 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:19.791946 22566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:19.795784 22566 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260620 07:09:19.797788 22582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:19.798957 22566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:19.799070 22566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "8b1e5632bf2b46e08e264332aa2bfbf2"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "06267e8e25b61dde880df18223aef343"
server_key_iv: "56c60a00d227440bdb1dfcdfab6b03f0"
server_key_version: "encryptionkey@0"
I20260620 07:09:19.799681 22566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:19.819836 22566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:19.822613 22566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:19.822783 22566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:19.829352 22566 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44247
I20260620 07:09:19.829619 22634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44247 every 8 connection(s)
I20260620 07:09:19.830386 22566 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:19.834573 22635 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:09:19.836202 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap starting.
I20260620 07:09:19.839673 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22566
I20260620 07:09:19.840049 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:35621
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=33119
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:19.852728 22635 log.cc:824] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:19.861438 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:19.861796 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap complete.
I20260620 07:09:19.865688 22635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:19.866080 22635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Initialized, Role: FOLLOWER
I20260620 07:09:19.866564 22635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:19.866685 22635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:19.866783 22635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:19.866909 22635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:09:19.868961 22635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:19.869282 22635 leader_election.cc:302] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b1e5632bf2b46e08e264332aa2bfbf2; no voters:
I20260620 07:09:19.869577 22635 leader_election.cc:288] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 07:09:19.869673 22639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 07:09:19.869867 22639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 LEADER]: Becoming Leader. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Running, Role: LEADER
I20260620 07:09:19.870223 22639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:19.870800 22635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:19.871626 22641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b1e5632bf2b46e08e264332aa2bfbf2. Latest consensus state: current_term: 2 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:19.871740 22641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:19.871979 22640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:19.872045 22640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:19.872076 22648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:19.876698 22648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:19.877367 22648 catalog_manager.cc:1259] Loaded cluster ID: dad8e39c94c44677b6006f5c32f76314
I20260620 07:09:19.877432 22648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:19.878770 22648 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 07:09:19.883555 22648 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 07:09:19.888767 22648 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:19.889515 22648 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Loaded TSK: 0
I20260620 07:09:19.890690 22648 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:09:19.950302 22637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:19.950558 22637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:19.950616 22637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:19.954643 22637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:19.954697 22637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:19.954774 22637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:19.959861 22637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:35621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33119
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:19.960757 22637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:19.961665 22637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:19.970476 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:19.970443 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:19.970448 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:19.970893 22637 server_base.cc:1034] running on GCE node
I20260620 07:09:19.971343 22637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:19.972098 22637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:19.973315 22637 hybrid_clock.cc:648] HybridClock initialized: now 1781939359973294 us; error 34 us; skew 500 ppm
Jun 20 07:09:19 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:19.976660 22637 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:19.977759 22637 webserver.cc:460] Webserver started at http://127.8.24.1:33119/ using document root <none> and password file <none>
I20260620 07:09:19.978447 22637 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:19.978556 22637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:19.983503 22637 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:19.986016 22672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:19.987511 22637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:09:19.987629 22637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "98b3cacc0a37ae43d2a3c45195d77f96"
server_key_iv: "f2ecbda28f3f823b952be6fc5e95293f"
server_key_version: "encryptionkey@0"
I20260620 07:09:19.988317 22637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:20.007714 22637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:20.010675 22637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:20.010843 22637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:20.011426 22637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:20.012535 22679 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 07:09:20.016381 22637 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 07:09:20.016433 22637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 07:09:20.016477 22637 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 07:09:20.018031 22637 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 07:09:20.018093 22637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 07:09:20.018359 22679 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap starting.
I20260620 07:09:20.028173 22637 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35621
I20260620 07:09:20.028206 22786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35621 every 8 connection(s)
I20260620 07:09:20.029362 22637 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:20.036044 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22637
I20260620 07:09:20.036441 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:39627
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=33733
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:20.038273 22679 log.cc:824] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Log is configured to *not* fsync() on all Append() calls
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:20.045662 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.031264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:58641)
0620 07:09:20.031926 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.031976 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.032740 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.034119 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.034125 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.034448 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:20.034901 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.034915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.036779 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.036782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.037995 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.038005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.038707 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.038711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.038888 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.039475 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:20.039494 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:20.040863 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:20.042756 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.042761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.042774 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.043016 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.043333 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.043336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.043338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.043505 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.044036 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:20.044041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:20.044287 (+ 246us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.044441 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.044760 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":153,"threads_started":1}
I20260620 07:09:20.045650 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.031377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58641 (local address 127.8.24.62:44247)
0620 07:09:20.031744 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.031762 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.032106 (+ 344us) server_negotiation.cc:407] Connection header received
0620 07:09:20.033211 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.033229 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.033716 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.034100 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:20.035081 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.036630 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.038113 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.038584 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.038730 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.041045 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:20.041051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:20.041064 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:20.041091 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:20.042644 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.043121 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.043126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.043130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.043245 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.043593 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.043596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.043598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.043927 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:20.044042 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.044606 (+ 564us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.044870 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":131,"threads_started":1}
I20260620 07:09:20.046880 22679 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:20.047281 22679 tablet_bootstrap.cc:492] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap complete.
I20260620 07:09:20.048552 22787 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:20.048621 22679 ts_tablet_manager.cc:1397] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent bootstrapping tablet: real 0.030s user 0.029s sys 0.000s
I20260620 07:09:20.048781 22787 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:20.049238 22787 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:20.051517 22599 ts_manager.cc:194] Registered new tserver with Master: 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621)
I20260620 07:09:20.051487 22679 raft_consensus.cc:348] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:20.051998 22679 raft_consensus.cc:729] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcf21e68e0e4606915a5fed7c588cc7, State: Initialized, Role: FOLLOWER
I20260620 07:09:20.052493 22679 consensus_queue.cc:260] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } }
I20260620 07:09:20.053480 22679 ts_tablet_manager.cc:1428] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:09:20.055025 22599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:58641
I20260620 07:09:20.057411 22787 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:09:20.064949 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.054375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35621 (local address 127.0.0.1:55148)
0620 07:09:20.054784 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.054834 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.054956 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.056221 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.056227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.056647 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:20.056771 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.056779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.057959 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.057963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.058928 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.058939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.059476 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.059494 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.059608 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.060099 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:20.060113 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:20.061565 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:20.063416 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.063426 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.063478 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.063697 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.063975 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.063978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.063979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.064050 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.064474 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:20.064478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:20.064578 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.064681 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.064821 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
I20260620 07:09:20.065074 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.054995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.8.24.1:35621)
0620 07:09:20.055825 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.055834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.055852 (+ 18us) server_negotiation.cc:407] Connection header received
0620 07:09:20.055902 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.055905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.056074 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.056202 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:20.056892 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.057853 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.059046 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.059368 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.059478 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.061688 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:20.061697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:20.061708 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:20.061734 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:20.063292 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.063792 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.063798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.063800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.063890 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.064120 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.064123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.064125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.064385 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:20.064504 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.064769 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.064922 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":116,"threads_started":1}
I20260620 07:09:20.066009 22721 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fbbc28ebd34c28b009fc6838df41d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:19 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55148
I20260620 07:09:20.066756 22796 tablet_replica.cc:330] stopping tablet replica
I20260620 07:09:20.067009 22796 raft_consensus.cc:2229] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:20.067229 22796 raft_consensus.cc:2258] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:20.068126 22796 ts_tablet_manager.cc:1905] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:20.071457 22796 ts_tablet_manager.cc:1918] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:20.071552 22796 log.cc:1196] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/c6fbbc28ebd34c28b009fc6838df41d3
I20260620 07:09:20.071769 22796 ts_tablet_manager.cc:1939] T c6fbbc28ebd34c28b009fc6838df41d3 P 1bcf21e68e0e4606915a5fed7c588cc7: Deleting consensus metadata
I20260620 07:09:20.073351 22586 catalog_manager.cc:4841] TS 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621): tablet c6fbbc28ebd34c28b009fc6838df41d3 (table test-table [id=1ab00a5c58db47d08450fcc55679aaed]) successfully deleted
W20260620 07:09:20.150475 22791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:20.150744 22791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:20.150786 22791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:20.154903 22791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:20.154978 22791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:20.155054 22791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:20.160444 22791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:39627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=33733
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:20.161495 22791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:20.162495 22791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:20.171543 22801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.171543 22802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.171571 22804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:20.171713 22791 server_base.cc:1034] running on GCE node
I20260620 07:09:20.172278 22791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:20.172920 22791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:20.174080 22791 hybrid_clock.cc:648] HybridClock initialized: now 1781939360174059 us; error 34 us; skew 500 ppm
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:20.176672 22791 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:20.177534 22791 webserver.cc:460] Webserver started at http://127.8.24.2:33733/ using document root <none> and password file <none>
I20260620 07:09:20.178089 22791 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:20.178171 22791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:20.181998 22791 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:20.184011 22811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.185189 22791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:20.185300 22791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "d0214eb22d87398c73346d8d08237f34"
server_key_iv: "ac913e398e2782f119fd9e82e0d72413"
server_key_version: "encryptionkey@0"
I20260620 07:09:20.185827 22791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:20.216662 22791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:20.219353 22791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:20.219547 22791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:20.220125 22791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:20.220923 22791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:20.220971 22791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.221021 22791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:20.221048 22791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.229455 22791 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39627
I20260620 07:09:20.229480 22924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39627 every 8 connection(s)
I20260620 07:09:20.230410 22791 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:20.233026 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22791
I20260620 07:09:20.233493 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:40011
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=34759
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:20.244190 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.232133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48861 (local address 127.8.24.62:44247)
0620 07:09:20.232262 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.232266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.233157 (+ 891us) server_negotiation.cc:407] Connection header received
0620 07:09:20.234782 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.234785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.234841 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.234923 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:20.236114 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.236877 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.238006 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.238369 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.238483 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.240715 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:20.240720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:20.240723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:20.240756 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:20.242255 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.242706 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.242710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.242712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.242806 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.243082 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.243085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.243087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.243315 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:20.243428 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.243892 (+ 464us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.244021 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:09:20.244880 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.232512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:48861)
0620 07:09:20.232994 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.233053 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.234521 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.235111 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.235118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.235493 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:20.235975 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.235989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.236980 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.236982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.237889 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.237898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.238497 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.238500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.238640 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.239164 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:20.239180 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:20.240541 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:20.242360 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.242364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.242376 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.242615 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.242885 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.242887 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.242888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.243002 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.243411 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:20.243415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:20.243675 (+ 260us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.243824 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.244061 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260620 07:09:20.245903 22925 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:20.246125 22925 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:20.246562 22925 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:20.247504 22599 ts_manager.cc:194] Registered new tserver with Master: 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
I20260620 07:09:20.248035 22599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:48861
W20260620 07:09:20.341632 22928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:20.341921 22928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:20.341982 22928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:20.346081 22928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:20.346158 22928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:20.346242 22928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:20.351537 22928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:40011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=34759
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:20.352577 22928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:20.353607 22928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:20.362622 22934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.362620 22936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.362623 22933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:20.362728 22928 server_base.cc:1034] running on GCE node
I20260620 07:09:20.363273 22928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:20.363910 22928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:20.365053 22928 hybrid_clock.cc:648] HybridClock initialized: now 1781939360365029 us; error 39 us; skew 500 ppm
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:20.367601 22928 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:20.368469 22928 webserver.cc:460] Webserver started at http://127.8.24.3:34759/ using document root <none> and password file <none>
I20260620 07:09:20.368969 22928 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:20.369050 22928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:20.372812 22928 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:09:20.374737 22943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.375955 22928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:20.376047 22928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "cae94afb87d042f1aab170f49f0609cd"
format_stamp: "Formatted at 2026-06-20 07:09:19 on dist-test-slave-wjw3"
server_key: "d2e51c652204d69c7c43a8e66de7a8e0"
server_key_iv: "24c1c02d14e508579a031895e92b92bb"
server_key_version: "encryptionkey@0"
I20260620 07:09:20.376590 22928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:20.394670 22928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:20.397619 22928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:20.397778 22928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:20.398309 22928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:20.399070 22928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:20.399114 22928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.399158 22928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:20.399186 22928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.407606 22928 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40011
I20260620 07:09:20.407771 23056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40011 every 8 connection(s)
I20260620 07:09:20.408887 22928 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:20.409791 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 22928
I20260620 07:09:20.418006 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.410938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.8.24.62:44247)
0620 07:09:20.411061 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.411066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.411083 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:09:20.411134 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.411137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.411186 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.411261 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:20.412064 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.412601 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.413479 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.413741 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.413832 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.415198 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:20.415201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:20.415205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:20.415231 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:20.416534 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.416921 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.416924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.416926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.416997 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.417248 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.417251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.417253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.417413 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:20.417603 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.417743 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.417844 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":43}
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:20.422407 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.411029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:40509 (local address 127.8.24.62:44247)
0620 07:09:20.411249 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.411253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.411990 (+ 737us) server_negotiation.cc:407] Connection header received
0620 07:09:20.413314 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.413317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.413390 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.413471 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:20.414765 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.415286 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.416620 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.416853 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.416936 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.419192 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:20.419195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:20.419198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:20.419221 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:20.420530 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.421013 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.421016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.421018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.421102 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:20.421396 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:20.421399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:20.421401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:20.421563 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:20.421644 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.422132 (+ 488us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.422245 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260620 07:09:20.423174 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.411346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:40509)
0620 07:09:20.411848 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.411900 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.413111 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.413671 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.413678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.414032 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:20.414625 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.414641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.415403 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.415406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.416502 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.416511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.416983 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.416987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.417144 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.417604 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:20.417620 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:20.419045 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:20.420634 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.420639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.420653 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.420926 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.421179 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:20.421182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:20.421184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:20.421314 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:20.421655 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:20.421662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:20.421890 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.422078 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.422384 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":116,"threads_started":1}
I20260620 07:09:20.424175 23057 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44247
I20260620 07:09:20.424383 23057 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:20.424822 23057 heartbeater.cc:511] Master 127.8.24.62:44247 requested a full tablet report, sending...
I20260620 07:09:20.425699 22599 ts_manager.cc:194] Registered new tserver with Master: cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011)
I20260620 07:09:20.426196 22599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:40509
I20260620 07:09:20.431510 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:20.435271 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.432061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.8.24.62:44247)
0620 07:09:20.432172 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.432176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.432349 (+ 173us) server_negotiation.cc:407] Connection header received
0620 07:09:20.432481 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.432484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.432586 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.432667 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:20.432845 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.433336 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.434153 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.434360 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.434427 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.434646 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:20.434999 (+ 353us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:20.435081 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.435126 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.435174 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:09:20.435839 22599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:09:20.438848 22599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:09:20.445956 22721 tablet_service.cc:1467] Processing CreateTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b (DEFAULT_TABLE table=test-table [id=bcc8195209cf437c8a4c2621fa31704a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:20.446599 22721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b01ab8311f45cf8d874ccfa573d52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:20.449360 22679 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap starting.
I20260620 07:09:20.449854 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.444931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40011 (local address 127.0.0.1:36768)
0620 07:09:20.445085 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.445098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.445168 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.445985 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.445989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.446009 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:20.446119 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.446125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.447706 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.447709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.448578 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.448588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.449430 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.449433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.449529 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.449589 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.449638 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.449717 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":36,"spinlock_wait_cycles":4992}
I20260620 07:09:20.450198 22679 tablet_bootstrap.cc:654] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:20.450515 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.445090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.8.24.3:40011)
0620 07:09:20.445455 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.445467 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.445490 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:09:20.445563 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.445568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.445806 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.445998 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:20.446250 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.447428 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.448691 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.449300 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.450093 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.450241 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.450291 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.450381 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":104,"threads_started":1}
I20260620 07:09:20.451018 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.445379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39627 (local address 127.0.0.1:35862)
0620 07:09:20.445949 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.445961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.446036 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.446501 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.446504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.446520 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:20.446618 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.446624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.447881 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.447884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.448994 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.449006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.450627 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.450631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.450723 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.450748 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.450823 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.450891 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260620 07:09:20.451057 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.445582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.8.24.2:39627)
0620 07:09:20.445898 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.445910 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.446137 (+ 227us) server_negotiation.cc:407] Connection header received
0620 07:09:20.446217 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.446222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.446385 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.446511 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:20.446735 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.447752 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.449125 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.450504 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.450610 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.450745 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.450847 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.450929 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260620 07:09:20.451798 22679 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: No bootstrap required, opened a new log
I20260620 07:09:20.451876 22679 ts_tablet_manager.cc:1397] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:20.452317 22679 raft_consensus.cc:348] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.452440 22679 raft_consensus.cc:374] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:20.452486 22679 raft_consensus.cc:729] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcf21e68e0e4606915a5fed7c588cc7, State: Initialized, Role: FOLLOWER
I20260620 07:09:20.452612 22679 consensus_queue.cc:260] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.452858 22679 ts_tablet_manager.cc:1428] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 07:09:20.453354 22991 tablet_service.cc:1467] Processing CreateTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b (DEFAULT_TABLE table=test-table [id=bcc8195209cf437c8a4c2621fa31704a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:20.453874 22859 tablet_service.cc:1467] Processing CreateTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b (DEFAULT_TABLE table=test-table [id=bcc8195209cf437c8a4c2621fa31704a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:20.454457 22991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b01ab8311f45cf8d874ccfa573d52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:20.454849 22859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b01ab8311f45cf8d874ccfa573d52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:20.460902 23070 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Bootstrap starting.
I20260620 07:09:20.460902 23071 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Bootstrap starting.
I20260620 07:09:20.462903 23070 tablet_bootstrap.cc:654] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:20.463219 23071 tablet_bootstrap.cc:654] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:20.463913 23070 log.cc:824] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:20.463914 23071 log.cc:824] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:20.465576 23071 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: No bootstrap required, opened a new log
I20260620 07:09:20.465590 23070 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: No bootstrap required, opened a new log
I20260620 07:09:20.465740 23070 ts_tablet_manager.cc:1397] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 07:09:20.465745 23071 ts_tablet_manager.cc:1397] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260620 07:09:20.468650 23070 raft_consensus.cc:348] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.468650 23071 raft_consensus.cc:348] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.468871 23071 raft_consensus.cc:374] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:20.468876 23070 raft_consensus.cc:374] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:20.468911 23071 raft_consensus.cc:729] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5bf3ea8ec04bde8d5e6483e164f7fb, State: Initialized, Role: FOLLOWER
I20260620 07:09:20.468931 23070 raft_consensus.cc:729] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae94afb87d042f1aab170f49f0609cd, State: Initialized, Role: FOLLOWER
I20260620 07:09:20.469297 23070 consensus_queue.cc:260] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.469309 23071 consensus_queue.cc:260] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.469879 23057 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
I20260620 07:09:20.469880 22925 heartbeater.cc:503] Master 127.8.24.62:44247 was elected leader, sending a full tablet report...
I20260620 07:09:20.470242 23071 ts_tablet_manager.cc:1428] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260620 07:09:20.470252 23070 ts_tablet_manager.cc:1428] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
W20260620 07:09:20.481264 22926 tablet.cc:2369] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:20.501715 23074 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:20.501915 23074 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.503065 23074 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627), 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621)
I20260620 07:09:20.506909 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.503276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39627 (local address 127.8.24.3:43469)
0620 07:09:20.503408 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.503423 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.503545 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.503824 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.503827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.503846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:20.503957 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.503962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.504848 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.504850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.505817 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.505827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.506601 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.506603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.506685 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.506704 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.506755 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.506808 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 07:09:20.506981 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.503423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:43469 (local address 127.8.24.2:39627)
0620 07:09:20.503579 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.503584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.503602 (+ 18us) server_negotiation.cc:407] Connection header received
0620 07:09:20.503673 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.503676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.503726 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.503800 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:20.504089 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.504752 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.505941 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.506492 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.506595 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.506661 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.506790 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.506859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 07:09:20.507627 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.503518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:39545 (local address 127.8.24.1:35621)
0620 07:09:20.503656 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.503660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.504257 (+ 597us) server_negotiation.cc:407] Connection header received
0620 07:09:20.504327 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.504330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.504390 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.504483 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:20.504680 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.505247 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.506526 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.507129 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.507229 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.507385 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.507459 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.507529 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:09:20.507658 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.503276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35621 (local address 127.8.24.3:39545)
0620 07:09:20.504170 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.504182 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.504259 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:20.504495 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:20.504497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:20.504508 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:20.504597 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.504601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.505359 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.505362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.506415 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:20.506424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.507241 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.507244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:20.507336 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.507360 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 07:09:20.507427 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:09:20.507520 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":589,"thread_start_us":115,"threads_started":1}
I20260620 07:09:20.507633 22879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b01ab8311f45cf8d874ccfa573d52b" candidate_uuid: "cae94afb87d042f1aab170f49f0609cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" is_pre_election: true
I20260620 07:09:20.507925 22879 raft_consensus.cc:2454] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae94afb87d042f1aab170f49f0609cd in term 0.
I20260620 07:09:20.508366 22944 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd; no voters:
I20260620 07:09:20.508576 22741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b01ab8311f45cf8d874ccfa573d52b" candidate_uuid: "cae94afb87d042f1aab170f49f0609cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" is_pre_election: true
I20260620 07:09:20.508627 23074 raft_consensus.cc:2781] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:20.508716 23074 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:20.508769 23074 raft_consensus.cc:3037] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:20.508873 22741 raft_consensus.cc:2454] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae94afb87d042f1aab170f49f0609cd in term 0.
I20260620 07:09:20.509804 23074 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.510095 23074 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [CANDIDATE]: Term 1 election: Requested vote from peers 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627), 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621)
I20260620 07:09:20.510383 22741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b01ab8311f45cf8d874ccfa573d52b" candidate_uuid: "cae94afb87d042f1aab170f49f0609cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
I20260620 07:09:20.510422 22879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b01ab8311f45cf8d874ccfa573d52b" candidate_uuid: "cae94afb87d042f1aab170f49f0609cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
I20260620 07:09:20.510509 22741 raft_consensus.cc:3037] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:20.510538 22879 raft_consensus.cc:3037] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:20.511518 22741 raft_consensus.cc:2454] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae94afb87d042f1aab170f49f0609cd in term 1.
I20260620 07:09:20.511580 22879 raft_consensus.cc:2454] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae94afb87d042f1aab170f49f0609cd in term 1.
I20260620 07:09:20.511775 22947 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7, cae94afb87d042f1aab170f49f0609cd; no voters:
I20260620 07:09:20.511978 23074 raft_consensus.cc:2781] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:20.512183 23074 raft_consensus.cc:686] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 1 LEADER]: Becoming Leader. State: Replica: cae94afb87d042f1aab170f49f0609cd, State: Running, Role: LEADER
I20260620 07:09:20.512517 23074 consensus_queue.cc:237] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:20.515482 22599 catalog_manager.cc:5495] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd reported cstate change: term changed from 0 to 1, leader changed from <none> to cae94afb87d042f1aab170f49f0609cd (127.8.24.3). New cstate: current_term: 1 leader_uuid: "cae94afb87d042f1aab170f49f0609cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } health_report { overall_health: UNKNOWN } } }
I20260620 07:09:20.528741 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.525872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.8.24.3:40011)
0620 07:09:20.525972 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:20.525975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:20.525988 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:09:20.526049 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:20.526051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:20.526103 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:20.526182 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:20.526362 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.526842 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.527615 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:20.527817 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:20.527887 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:20.528100 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:20.528475 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:20.528554 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:20.528599 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 07:09:20.528644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 07:09:20.530495 22788 tablet.cc:2369] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:20.534163 22741 raft_consensus.cc:1261] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Refusing update from remote peer cae94afb87d042f1aab170f49f0609cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:20.534168 22879 raft_consensus.cc:1261] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Refusing update from remote peer cae94afb87d042f1aab170f49f0609cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:20.534814 23077 consensus_queue.cc:1035] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:20.534983 23074 consensus_queue.cc:1035] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:20.539691 23080 mvcc.cc:205] Tried to move back new op lower bound from 7298823620741353472 to 7298823620661809152. Current Snapshot: MvccSnapshot[applied={T|T < 7298823620741353472}]
I20260620 07:09:20.548115 22598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:20.548260 22598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:20.550376 22598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Sending DeleteTablet for 3 replicas of tablet 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:20.550967 22991 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:20 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36768
I20260620 07:09:20.551067 22859 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:20 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35862
I20260620 07:09:20.551143 22721 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b01ab8311f45cf8d874ccfa573d52b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:20 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:55148
I20260620 07:09:20.551337 22796 tablet_replica.cc:330] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: stopping tablet replica
I20260620 07:09:20.551482 22796 raft_consensus.cc:2229] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:20.551632 22796 raft_consensus.cc:2258] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:20.551644 23091 tablet_replica.cc:330] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: stopping tablet replica
I20260620 07:09:20.551661 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22637
I20260620 07:09:20.551844 23090 tablet_replica.cc:330] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: stopping tablet replica
I20260620 07:09:20.552152 23090 raft_consensus.cc:2229] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:09:20.552392 23091 raft_consensus.cc:2229] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:20.552464 23090 raft_consensus.cc:2258] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:20.552672 23091 raft_consensus.cc:2258] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:20.553941 23091 ts_tablet_manager.cc:1905] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:20.553941 23090 ts_tablet_manager.cc:1905] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:20.556512 23091 ts_tablet_manager.cc:1918] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:20.556522 23090 ts_tablet_manager.cc:1918] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:20.556608 23090 log.cc:1196] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:20.556614 23091 log.cc:1196] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:20.556928 23091 ts_tablet_manager.cc:1939] T 73b01ab8311f45cf8d874ccfa573d52b P 5e5bf3ea8ec04bde8d5e6483e164f7fb: Deleting consensus metadata
I20260620 07:09:20.556928 23090 ts_tablet_manager.cc:1939] T 73b01ab8311f45cf8d874ccfa573d52b P cae94afb87d042f1aab170f49f0609cd: Deleting consensus metadata
I20260620 07:09:20.558023 22585 catalog_manager.cc:4841] TS cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): tablet 73b01ab8311f45cf8d874ccfa573d52b (table test-table [id=bcc8195209cf437c8a4c2621fa31704a]) successfully deleted
I20260620 07:09:20.558033 22583 catalog_manager.cc:4841] TS 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): tablet 73b01ab8311f45cf8d874ccfa573d52b (table test-table [id=bcc8195209cf437c8a4c2621fa31704a]) successfully deleted
I20260620 07:09:20.560200 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22791
W20260620 07:09:20.560211 22586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:35621 (error 108)
I20260620 07:09:20.564180 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:20.563926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:35621 (local address 127.0.0.1:55160)
0620 07:09:20.564043 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.564105 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 07:09:20.564410 22586 catalog_manager.cc:4584] TS 1bcf21e68e0e4606915a5fed7c588cc7 (127.8.24.1:35621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73b01ab8311f45cf8d874ccfa573d52b: Network error: Client connection negotiation failed: client connection to 127.8.24.1:35621: connect: Connection refused (error 111)
I20260620 07:09:20.568121 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22928
I20260620 07:09:20.573171 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 22566
I20260620 07:09:20.578202 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--webserver_interface=127.8.24.62
--webserver_port=44787
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:20.685714 23093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:20.685982 23093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:20.686028 23093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:20.690081 23093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:20.690143 23093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:20.690163 23093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:20.690192 23093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:20.690218 23093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:20.695567 23093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=44787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:20.696604 23093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:20.697599 23093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:20.705451 23098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.705451 23099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.705493 23101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:20.705822 23093 server_base.cc:1034] running on GCE node
I20260620 07:09:20.706331 23093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:20.707345 23093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:20.708487 23093 hybrid_clock.cc:648] HybridClock initialized: now 1781939360708459 us; error 41 us; skew 500 ppm
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:20.710929 23093 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:20.711793 23093 webserver.cc:460] Webserver started at http://127.8.24.62:44787/ using document root <none> and password file <none>
I20260620 07:09:20.712304 23093 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:20.712368 23093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:20.716084 23093 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:20.718011 23108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.719147 23093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:20.719251 23093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "8b1e5632bf2b46e08e264332aa2bfbf2"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "06267e8e25b61dde880df18223aef343"
server_key_iv: "56c60a00d227440bdb1dfcdfab6b03f0"
server_key_version: "encryptionkey@0"
I20260620 07:09:20.719790 23093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:20.738476 23093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:20.741322 23093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:20.741495 23093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:20.748240 23093 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44247
I20260620 07:09:20.748484 23160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44247 every 8 connection(s)
I20260620 07:09:20.749293 23093 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:20.753360 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23093
I20260620 07:09:20.753505 23161 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:09:20.753810 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:35621
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=33119
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:20.755265 23161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap starting.
I20260620 07:09:20.777803 23161 log.cc:824] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:20.791513 23161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:20.791934 23161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: Bootstrap complete.
I20260620 07:09:20.797252 23161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:20.797801 23161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Initialized, Role: FOLLOWER
I20260620 07:09:20.798358 23161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:20.798502 23161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:20.798568 23161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:20.798671 23161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 2 FOLLOWER]: Advancing to term 3
I20260620 07:09:20.801124 23161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:20.801488 23161 leader_election.cc:302] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b1e5632bf2b46e08e264332aa2bfbf2; no voters:
I20260620 07:09:20.801820 23161 leader_election.cc:288] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 07:09:20.801935 23165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 07:09:20.802178 23165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [term 3 LEADER]: Becoming Leader. State: Replica: 8b1e5632bf2b46e08e264332aa2bfbf2, State: Running, Role: LEADER
I20260620 07:09:20.802475 23165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } }
I20260620 07:09:20.803030 23161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:20.803889 23166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:20.803989 23166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:20.803853 23167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b1e5632bf2b46e08e264332aa2bfbf2. Latest consensus state: current_term: 3 leader_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1e5632bf2b46e08e264332aa2bfbf2" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44247 } } }
I20260620 07:09:20.804095 23167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2 [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:20.808331 23180 catalog_manager.cc:1259] Loaded cluster ID: dad8e39c94c44677b6006f5c32f76314
I20260620 07:09:20.808401 23180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: loading cluster ID for follower catalog manager: success
W20260620 07:09:20.809417 23180 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8b1e5632bf2b46e08e264332aa2bfbf2: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260620 07:09:20.809485 23180 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260620 07:09:20.809541 23174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:20.812204 23174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:20.812744 23174 catalog_manager.cc:1259] Loaded cluster ID: dad8e39c94c44677b6006f5c32f76314
I20260620 07:09:20.812808 23174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 07:09:20.813486 23174 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe78add59ad google::LogMessage::Fail() at ??:0
@ 0x7fe78add888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe78add54de google::LogMessage::Flush() at ??:0
@ 0x7fe78add6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe795714fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe795715d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe795712579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe795756640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe78d0273d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe78d027c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe78d0293f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe78d018512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe78e65c6db start_thread at ??:0
@ 0x7fe7898d361f clone at ??:0
W20260620 07:09:20.913123 23163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:20.913497 23163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:20.913563 23163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:20.919562 23163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:20.919641 23163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:20.919728 23163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:20.927356 23163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:35621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=33119
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:20.928835 23163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:20.930367 23163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:20.939733 23203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.939733 23202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:20.939822 23205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:20.940001 23163 server_base.cc:1034] running on GCE node
I20260620 07:09:20.940476 23163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:20.941112 23163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:20.942271 23163 hybrid_clock.cc:648] HybridClock initialized: now 1781939360942253 us; error 33 us; skew 500 ppm
Jun 20 07:09:20 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:20.944828 23163 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:20.945648 23163 webserver.cc:460] Webserver started at http://127.8.24.1:33119/ using document root <none> and password file <none>
I20260620 07:09:20.946177 23163 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:20.946235 23163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:20.950150 23163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260620 07:09:20.952360 23212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:20.953701 23163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:20.953819 23163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "1bcf21e68e0e4606915a5fed7c588cc7"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "98b3cacc0a37ae43d2a3c45195d77f96"
server_key_iv: "f2ecbda28f3f823b952be6fc5e95293f"
server_key_version: "encryptionkey@0"
I20260620 07:09:20.954398 23163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:20.964545 23163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:20.967543 23163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:20.967708 23163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:20.968264 23163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:20.969331 23219 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 07:09:20.974677 23163 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 07:09:20.974740 23163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260620 07:09:20.974803 23163 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 07:09:20.976495 23163 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 07:09:20.976539 23163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260620 07:09:20.976861 23219 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap starting.
I20260620 07:09:20.986110 23163 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:35621
I20260620 07:09:20.986356 23326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:35621 every 8 connection(s)
I20260620 07:09:20.987087 23163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:20.990731 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23163
I20260620 07:09:20.991125 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:39627
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=33733
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:20.999025 23219 log.cc:824] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:21.005843 23219 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:21.006178 23219 tablet_bootstrap.cc:492] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Bootstrap complete.
I20260620 07:09:21.007071 23219 ts_tablet_manager.cc:1397] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent bootstrapping tablet: real 0.030s user 0.020s sys 0.004s
I20260620 07:09:21.009450 23219 raft_consensus.cc:348] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:21.009865 23219 raft_consensus.cc:729] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcf21e68e0e4606915a5fed7c588cc7, State: Initialized, Role: FOLLOWER
I20260620 07:09:21.010329 23219 consensus_queue.cc:260] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:21.011219 23219 ts_tablet_manager.cc:1428] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260620 07:09:21.081897 23329 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:09:20.989027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:55905)
0620 07:09:20.989514 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:20.989578 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:20.990419 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:21.081162 (+ 90743us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
W20260620 07:09:21.082347 23327 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:44247 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:44247: Client connection negotiation failed: client connection to 127.8.24.62:44247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 07:09:21.083029 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.082791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:46361)
0620 07:09:21.082908 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.082976 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 07:09:21.083637 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.083427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:51445)
0620 07:09:21.083533 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.083586 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 07:09:21.083777 23327 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:09:21.127712 23330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:21.127974 23330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:21.128033 23330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:21.132043 23330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:21.132098 23330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:21.132176 23330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:21.137267 23330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:39627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=33733
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:21.138247 23330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:21.139236 23330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:21.148056 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:21.148108 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:21.148077 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:21.148377 23330 server_base.cc:1034] running on GCE node
I20260620 07:09:21.148777 23330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:21.149343 23330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:21.150492 23330 hybrid_clock.cc:648] HybridClock initialized: now 1781939361150480 us; error 25 us; skew 500 ppm
Jun 20 07:09:21 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:21.153043 23330 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:21.153905 23330 webserver.cc:460] Webserver started at http://127.8.24.2:33733/ using document root <none> and password file <none>
I20260620 07:09:21.154415 23330 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:21.154477 23330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:21.158197 23330 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:21.160197 23347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.161331 23330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:21.161427 23330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb"
format_stamp: "Formatted at 2026-06-20 07:09:18 on dist-test-slave-wjw3"
server_key: "d0214eb22d87398c73346d8d08237f34"
server_key_iv: "ac913e398e2782f119fd9e82e0d72413"
server_key_version: "encryptionkey@0"
I20260620 07:09:21.161959 23330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:21.173662 23330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:21.176808 23330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:21.176980 23330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:21.177565 23330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:21.178372 23330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:21.178419 23330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.178463 23330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:21.178491 23330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.186714 23330 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39627
I20260620 07:09:21.186740 23460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39627 every 8 connection(s)
I20260620 07:09:21.187736 23330 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:21.189775 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23330
I20260620 07:09:21.190155 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:40011
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:44247
--webserver_port=34759
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:21.191529 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.189763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:33549)
0620 07:09:21.190252 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.190803 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":131,"threads_started":1}
W20260620 07:09:21.191934 23461 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:44247 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:44247: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
I20260620 07:09:21.192512 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.192299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:60155)
0620 07:09:21.192401 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.192447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 07:09:21.193065 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.192870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:54353)
0620 07:09:21.192961 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.193005 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 07:09:21.193205 23461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:09:21.296048 23464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:21.296294 23464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:21.296336 23464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:21.300185 23464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:21.300236 23464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:21.300289 23464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:21.305264 23464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:40011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=34759
--tserver_master_addrs=127.8.24.62:44247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:21.306188 23464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:21.307111 23464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:21.315997 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:21.316154 23464 server_base.cc:1034] running on GCE node
W20260620 07:09:21.316171 23472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:21.316041 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:21.316622 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:21.317186 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:21.318367 23464 hybrid_clock.cc:648] HybridClock initialized: now 1781939361318342 us; error 45 us; skew 500 ppm
Jun 20 07:09:21 dist-test-slave-wjw3 krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939361, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:21.320864 23464 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:21.321697 23464 webserver.cc:460] Webserver started at http://127.8.24.3:34759/ using document root <none> and password file <none>
I20260620 07:09:21.322220 23464 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:21.322280 23464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:21.325908 23464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:21.327813 23479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.328979 23464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:21.329092 23464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "cae94afb87d042f1aab170f49f0609cd"
format_stamp: "Formatted at 2026-06-20 07:09:19 on dist-test-slave-wjw3"
server_key: "d2e51c652204d69c7c43a8e66de7a8e0"
server_key_iv: "24c1c02d14e508579a031895e92b92bb"
server_key_version: "encryptionkey@0"
I20260620 07:09:21.329618 23464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:21.362339 23464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:21.365079 23464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:21.365247 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:21.365913 23464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:21.366740 23464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:21.366786 23464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.366830 23464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:21.366858 23464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:21.374933 23464 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:40011
I20260620 07:09:21.374962 23592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:40011 every 8 connection(s)
I20260620 07:09:21.375938 23464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:21.379552 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.377915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:58293)
0620 07:09:21.378334 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.378801 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
W20260620 07:09:21.379942 23593 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:44247 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:44247: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
I20260620 07:09:21.380467 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.380277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:47189)
0620 07:09:21.380371 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.380414 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 07:09:21.380963 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:21.380793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:35157)
0620 07:09:21.380877 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:21.380916 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 07:09:21.381117 23593 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 07:09:21.386165 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23464
I20260620 07:09:22.084729 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.084294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:36175)
0620 07:09:22.084551 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:22.084618 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260620 07:09:22.194008 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.193666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:38039)
0620 07:09:22.193865 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:22.193911 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260620 07:09:22.287489 23599 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:22.287751 23599 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:22.288825 23599 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011), 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
Jun 20 07:09:22 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.3@KRBTEST.COM
Jun 20 07:09:22 dist-test-slave-wjw3 krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939360, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.2@KRBTEST.COM
I20260620 07:09:22.309146 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.289251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:38723 (local address 127.8.24.3:40011)
0620 07:09:22.289567 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:22.289583 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:22.289629 (+ 46us) server_negotiation.cc:407] Connection header received
0620 07:09:22.290480 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:22.290490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:22.290956 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:22.291290 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:22.293024 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.294612 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.296407 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.297118 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.297277 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:22.304153 (+ 6876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:22.304158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:22.304171 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:22.304200 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:22.306025 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306741 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:22.306747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:22.306753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:22.306898 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:22.307285 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:22.307288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:22.307289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:22.307836 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:22.307971 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:22.308673 (+ 702us) server_negotiation.cc:299] Negotiation successful
0620 07:09:22.308961 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260620 07:09:22.309144 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.289235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:33761 (local address 127.8.24.2:39627)
0620 07:09:22.289657 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:22.289677 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:22.289705 (+ 28us) server_negotiation.cc:407] Connection header received
0620 07:09:22.290486 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:22.290492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:22.290956 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:22.291289 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:22.293024 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.294612 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.296407 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.297118 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.297277 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:22.304419 (+ 7142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:22.304426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:22.304442 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:22.304477 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:22.306087 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306741 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:22.306747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:22.306753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:22.306905 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:22.307285 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:22.307288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:22.307290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:22.307836 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:22.307971 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:22.308673 (+ 702us) server_negotiation.cc:299] Negotiation successful
0620 07:09:22.308961 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":105,"threads_started":1}
I20260620 07:09:22.309180 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.289020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:40011 (local address 127.8.24.1:38723)
0620 07:09:22.289197 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:22.289210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:22.289310 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:22.291396 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:22.291402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:22.291993 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:22.292853 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:22.292874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.294761 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.294764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:22.296304 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:22.296315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.297226 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.297229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:22.297532 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:22.302355 (+ 4823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:22.302371 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:22.303946 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:22.306138 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:22.306160 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:22.306626 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:22.306986 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:22.306990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:22.307191 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:22.307942 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:22.307949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:22.308323 (+ 374us) client_negotiation.cc:769] Sending connection context
0620 07:09:22.308534 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 07:09:22.308821 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":442,"spinlock_wait_cycles":26752}
I20260620 07:09:22.309412 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.289065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39627 (local address 127.8.24.1:33761)
0620 07:09:22.289367 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:22.289379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:22.289471 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:22.291396 (+ 1925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:22.291402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:22.291993 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:22.292853 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:22.292874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.294765 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.294768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:22.296305 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:22.296315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:22.297226 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:22.297229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:22.297531 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:22.302363 (+ 4832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:22.302380 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:22.304233 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:22.306166 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:22.306171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:22.306626 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:22.306986 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:22.306988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:22.306990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:22.307191 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:22.307942 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:22.307949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:22.308323 (+ 374us) client_negotiation.cc:769] Sending connection context
0620 07:09:22.308534 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 07:09:22.309159 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":304,"spinlock_wait_cycles":25472,"thread_start_us":85,"threads_started":1}
W20260620 07:09:22.311817 23213 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
W20260620 07:09:22.311946 23215 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:22.312031 23215 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7; no voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd
I20260620 07:09:22.312269 23599 raft_consensus.cc:2726] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:09:22.381966 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:22.381604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:43005)
0620 07:09:22.381826 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:22.381877 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260620 07:09:23.085834 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:23.085387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:45351)
0620 07:09:23.085680 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:23.085738 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":74,"threads_started":1}
I20260620 07:09:23.195030 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:23.194634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:54331)
0620 07:09:23.194880 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:23.194938 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":80,"threads_started":1}
I20260620 07:09:23.383076 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:23.382653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:48003)
0620 07:09:23.382906 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:23.382964 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260620 07:09:23.936501 23607 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:23.936635 23607 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:23.937011 23607 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011), 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
W20260620 07:09:23.937759 23215 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
W20260620 07:09:23.937871 23213 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:23.938021 23213 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7; no voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd
I20260620 07:09:23.938169 23607 raft_consensus.cc:2726] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:09:24.086894 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:24.086512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:33733)
0620 07:09:24.086740 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:24.086790 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260620 07:09:24.196080 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:24.195683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:46817)
0620 07:09:24.195924 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:24.195977 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
I20260620 07:09:24.384229 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:24.383818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:55735)
0620 07:09:24.384070 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:24.384131 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260620 07:09:25.087998 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:25.087604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:32883)
0620 07:09:25.087846 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:25.087901 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260620 07:09:25.197100 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:25.196713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:37963)
0620 07:09:25.196948 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:25.197005 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260620 07:09:25.385286 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:25.384888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:51473)
0620 07:09:25.385137 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:25.385195 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260620 07:09:26.089251 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:26.088809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:33757)
0620 07:09:26.089090 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:26.089145 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260620 07:09:26.198148 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:26.197731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:56611)
0620 07:09:26.197988 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:26.198055 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260620 07:09:26.386301 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:26.385938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:36029)
0620 07:09:26.386157 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:26.386209 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260620 07:09:26.981611 23617 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:26.981736 23617 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:26.982105 23617 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011), 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
W20260620 07:09:26.982822 23215 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
W20260620 07:09:26.982933 23213 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:26.983037 23213 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7; no voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd
I20260620 07:09:26.983181 23617 raft_consensus.cc:2726] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:09:27.090334 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:27.089923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:34203)
0620 07:09:27.090178 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:27.090237 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260620 07:09:27.199306 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:27.198916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:59065)
0620 07:09:27.199160 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:27.199214 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260620 07:09:27.387322 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:27.386955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:54459)
0620 07:09:27.387184 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:27.387235 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260620 07:09:28.091427 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:28.091026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:48135)
0620 07:09:28.091273 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:28.091328 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
I20260620 07:09:28.200342 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:28.199944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:59613)
0620 07:09:28.200202 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:28.200254 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":65,"threads_started":1}
I20260620 07:09:28.388517 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:28.388050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:53063)
0620 07:09:28.388340 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:28.388404 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":82,"threads_started":1}
I20260620 07:09:29.092623 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:29.092158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:47329)
0620 07:09:29.092453 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:29.092515 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260620 07:09:29.201345 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:29.200981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:36845)
0620 07:09:29.201205 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:29.201258 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":65,"threads_started":1}
I20260620 07:09:29.389508 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:29.389140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:50981)
0620 07:09:29.389365 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:29.389415 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260620 07:09:30.093797 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:30.093388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:60629)
0620 07:09:30.093650 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:30.093703 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":91,"threads_started":1}
I20260620 07:09:30.202360 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:30.201974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:47895)
0620 07:09:30.202215 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:30.202268 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260620 07:09:30.390515 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:30.390136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:44085)
0620 07:09:30.390373 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:30.390425 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260620 07:09:31.099884 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:31.099373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:56819)
0620 07:09:31.099692 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:31.099758 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
I20260620 07:09:31.203424 23631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:31.203031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:48589)
0620 07:09:31.203280 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:31.203331 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260620 07:09:31.303202 23632 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:31.303323 23632 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:31.303700 23632 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011), 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
W20260620 07:09:31.304402 23215 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
W20260620 07:09:31.304530 23213 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:31.304615 23213 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7; no voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd
I20260620 07:09:31.304741 23632 raft_consensus.cc:2726] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:09:31.391516 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:31.391128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:49407)
0620 07:09:31.391335 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:31.391386 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260620 07:09:32.101080 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:32.100700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:54399)
0620 07:09:32.100932 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:32.100985 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260620 07:09:32.204454 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:32.204076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:36635)
0620 07:09:32.204305 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:32.204362 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260620 07:09:32.392555 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:32.392149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:45521)
0620 07:09:32.392385 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:32.392439 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260620 07:09:33.102147 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:33.101710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:38479)
0620 07:09:33.101989 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:33.102043 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260620 07:09:33.205569 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:33.205137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:39551)
0620 07:09:33.205405 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:33.205474 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260620 07:09:33.393597 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:33.393198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:46901)
0620 07:09:33.393445 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:33.393503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260620 07:09:34.103255 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:34.102797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:38423)
0620 07:09:34.103103 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:34.103160 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260620 07:09:34.206630 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:34.206235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:42977)
0620 07:09:34.206482 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:34.206536 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260620 07:09:34.394721 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:34.394314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:45571)
0620 07:09:34.394568 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:34.394622 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260620 07:09:35.104310 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:35.103911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:37721)
0620 07:09:35.104164 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:35.104214 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260620 07:09:35.207798 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:35.207265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:48039)
0620 07:09:35.207626 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:35.207686 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":83,"threads_started":1}
I20260620 07:09:35.395778 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:35.395356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:45235)
0620 07:09:35.395621 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:35.395673 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260620 07:09:35.675887 23646 raft_consensus.cc:482] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:35.676015 23646 raft_consensus.cc:504] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cae94afb87d042f1aab170f49f0609cd" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 40011 } } peers { permanent_uuid: "5e5bf3ea8ec04bde8d5e6483e164f7fb" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39627 } } peers { permanent_uuid: "1bcf21e68e0e4606915a5fed7c588cc7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 35621 } }
I20260620 07:09:35.676394 23646 leader_election.cc:288] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011), 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627)
W20260620 07:09:35.677052 23213 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5bf3ea8ec04bde8d5e6483e164f7fb (127.8.24.2:39627): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
W20260620 07:09:35.677181 23215 leader_election.cc:341] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cae94afb87d042f1aab170f49f0609cd (127.8.24.3:40011): Not found: Tablet not found: 73b01ab8311f45cf8d874ccfa573d52b
I20260620 07:09:35.677275 23215 leader_election.cc:302] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcf21e68e0e4606915a5fed7c588cc7; no voters: 5e5bf3ea8ec04bde8d5e6483e164f7fb, cae94afb87d042f1aab170f49f0609cd
I20260620 07:09:35.677420 23646 raft_consensus.cc:2726] T 73b01ab8311f45cf8d874ccfa573d52b P 1bcf21e68e0e4606915a5fed7c588cc7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:09:36.105504 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:36.105121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.1:59143)
0620 07:09:36.105363 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:36.105413 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260620 07:09:36.208981 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:36.208603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.2:42555)
0620 07:09:36.208837 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:36.208892 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260620 07:09:36.391672 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23163
I20260620 07:09:36.396692 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23330
I20260620 07:09:36.397387 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:36.396576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44247 (local address 127.8.24.3:37369)
0620 07:09:36.396804 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:36.397273 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:44247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260620 07:09:36.401185 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23464
2026-06-20T07:09:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19131 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:09:36 dist-test-slave-wjw3 krb5kdc[23652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:09:36 dist-test-slave-wjw3 krb5kdc[23652](info): set up 2 sockets
Jun 20 07:09:36 dist-test-slave-wjw3 krb5kdc[23652](info): commencing operation
krb5kdc: starting...
W20260620 07:09:37.359390 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:09:37 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:09:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:09:37.470191 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:37.574839 23665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:37.575098 23665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:37.575137 23665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:37.579066 23665 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 07:09:37.579113 23665 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 07:09:37.579128 23665 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 07:09:37.579142 23665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:37.579172 23665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:37.579202 23665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:37.579228 23665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:37.579253 23665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:37.584376 23665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:37.585335 23665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:37.586305 23665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:09:37.595034 23665 server_base.cc:1034] running on GCE node
W20260620 07:09:37.594988 23673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:37.594993 23671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:37.594988 23670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:37.595881 23665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:37.596956 23665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:37.598102 23665 hybrid_clock.cc:648] HybridClock initialized: now 1781939377598077 us; error 40 us; skew 500 ppm
Jun 20 07:09:37 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:37.600649 23665 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:37.601554 23665 webserver.cc:460] Webserver started at http://127.8.24.62:45449/ using document root <none> and password file <none>
I20260620 07:09:37.602083 23665 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:37.602149 23665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:37.602334 23665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:37.604411 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "ae76e7ce9b7f4b58b56f92b907238a5c"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "17dd728ecc187cddc37684722b64ad81"
server_key_iv: "abcde1b99f5070621e35d9e9a676fc5e"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.604928 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "ae76e7ce9b7f4b58b56f92b907238a5c"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "17dd728ecc187cddc37684722b64ad81"
server_key_iv: "abcde1b99f5070621e35d9e9a676fc5e"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.608448 23665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:37.610656 23680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:37.611881 23665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:37.611996 23665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ae76e7ce9b7f4b58b56f92b907238a5c"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "17dd728ecc187cddc37684722b64ad81"
server_key_iv: "abcde1b99f5070621e35d9e9a676fc5e"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.612085 23665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:37.623560 23665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:37.626215 23665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:37.626387 23665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:37.632840 23665 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38805
I20260620 07:09:37.633097 23732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38805 every 8 connection(s)
I20260620 07:09:37.633771 23665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:37.634758 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23665
I20260620 07:09:37.634859 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:09:37.635136 8288 external_mini_cluster.cc:1428] Setting key 3df758a4e63256f7e95cae58014e87ab
I20260620 07:09:37.636687 23733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:37.643141 23733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap starting.
Jun 20 07:09:37 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:37.645601 23733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:37.646454 23733 log.cc:824] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:37.648675 23733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: No bootstrap required, opened a new log
I20260620 07:09:37.649706 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:37.636561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.8.24.62:38805)
0620 07:09:37.636969 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:37.636988 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:37.637029 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:09:37.637865 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:37.637886 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:37.638288 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:37.638652 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:37.639293 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.640765 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.641628 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.642325 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.642491 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:37.644957 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:37.644963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:37.644977 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:37.645009 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:37.646845 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:37.647291 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:37.647297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:37.647302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:37.647470 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:37.647691 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:37.647693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:37.647695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:37.648162 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:37.648340 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:37.648519 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 07:09:37.648798 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260620 07:09:37.652159 23733 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:37.652362 23733 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:37.652410 23733 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Initialized, Role: FOLLOWER
I20260620 07:09:37.652748 23733 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:37.652851 23733 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:37.652900 23733 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:37.652967 23733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:37.653774 23733 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:37.654098 23733 leader_election.cc:302] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae76e7ce9b7f4b58b56f92b907238a5c; no voters:
I20260620 07:09:37.654366 23733 leader_election.cc:288] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:09:37.654554 23738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:37.654750 23738 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 LEADER]: Becoming Leader. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Running, Role: LEADER
I20260620 07:09:37.655014 23738 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:37.655573 23733 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:37.656641 23739 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:09:37.656621 23740 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae76e7ce9b7f4b58b56f92b907238a5c. Latest consensus state: current_term: 1 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:09:37.656785 23740 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:37.656783 23739 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:37.657790 23748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:37.660284 23748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:37.664774 23748 catalog_manager.cc:1347] Generated new cluster ID: 9301fe9fb3cf4226847b097871078432
I20260620 07:09:37.664832 23748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:37.674556 23748 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:09:37.675472 23748 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:37.685518 23748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Generated new TSK 0
I20260620 07:09:37.686340 23748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:09:37.717682 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:37.823331 23759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:37.823613 23759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:37.823665 23759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:37.827615 23759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:37.827672 23759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:37.827739 23759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:37.832866 23759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:37.833864 23759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:37.834830 23759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:37.843731 23765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:37.843813 23764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:37.843899 23759 server_base.cc:1034] running on GCE node
W20260620 07:09:37.843770 23767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:37.844381 23759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:37.844951 23759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:37.846091 23759 hybrid_clock.cc:648] HybridClock initialized: now 1781939377846075 us; error 30 us; skew 500 ppm
Jun 20 07:09:37 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:37.848555 23759 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:37.849323 23759 webserver.cc:460] Webserver started at http://127.8.24.1:38165/ using document root <none> and password file <none>
I20260620 07:09:37.849815 23759 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:37.849877 23759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:37.850060 23759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:37.851976 23759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "f831000e02fd4e2da033e2c6baf548ab"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "d76744b8ba4ee9655fe261046cdb8a7b"
server_key_iv: "064f1160d239e124e99376bfdeea632a"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.852483 23759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "f831000e02fd4e2da033e2c6baf548ab"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "d76744b8ba4ee9655fe261046cdb8a7b"
server_key_iv: "064f1160d239e124e99376bfdeea632a"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.855855 23759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:37.858089 23774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:37.859169 23759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:37.859278 23759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "f831000e02fd4e2da033e2c6baf548ab"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "d76744b8ba4ee9655fe261046cdb8a7b"
server_key_iv: "064f1160d239e124e99376bfdeea632a"
server_key_version: "encryptionkey@0"
I20260620 07:09:37.859375 23759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:37.875982 23759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:37.878599 23759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:37.878763 23759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:37.879308 23759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:37.880098 23759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:37.880146 23759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:37.880196 23759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:37.880224 23759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:37.888285 23759 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:36761
I20260620 07:09:37.888311 23887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:36761 every 8 connection(s)
I20260620 07:09:37.889098 23759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:37.893352 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23759
I20260620 07:09:37.893463 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:09:37.893697 8288 external_mini_cluster.cc:1428] Setting key fd4d6e929064c34f75c84b2e46f1a051
Jun 20 07:09:37 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:37.902112 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:37.890770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58051 (local address 127.8.24.62:38805)
0620 07:09:37.890898 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:37.890902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:37.891675 (+ 773us) server_negotiation.cc:407] Connection header received
0620 07:09:37.892896 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:37.892899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:37.892962 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:37.893034 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:37.894363 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.895102 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.896253 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.896491 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.896577 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:37.898818 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:37.898822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:37.898825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:37.898854 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:37.900302 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:37.900737 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:37.900740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:37.900742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:37.900828 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:37.901103 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:37.901106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:37.901108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:37.901318 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:37.901413 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:37.901830 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 07:09:37.901954 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:09:37.902781 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:37.891099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:58051)
0620 07:09:37.891548 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:37.891590 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:37.892702 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:37.893209 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:37.893226 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:37.893611 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:37.894224 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:37.894241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.895229 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.895232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:37.896141 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:37.896150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:37.896624 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:37.896627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:37.896784 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:37.897295 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:37.897311 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:37.898660 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:37.900406 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:37.900411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:37.900422 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:37.900650 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:37.900908 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:37.900910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:37.900912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:37.901027 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:37.901405 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:37.901410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:37.901618 (+ 208us) client_negotiation.cc:769] Sending connection context
0620 07:09:37.901757 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 07:09:37.901996 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":111,"threads_started":1}
I20260620 07:09:37.903927 23888 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:37.904152 23888 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:09:37.904592 23888 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:37.906023 23697 ts_manager.cc:194] Registered new tserver with Master: f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761)
I20260620 07:09:37.907084 23697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:58051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:09:37.917812 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:38.024165 23893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:38.024396 23893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:38.024439 23893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:38.028354 23893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:38.028403 23893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:38.028472 23893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:38.033459 23893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:38.034379 23893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:38.035359 23893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:38.044080 23899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:38.044108 23898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:38.044286 23893 server_base.cc:1034] running on GCE node
W20260620 07:09:38.044377 23901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:38.044795 23893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:38.045385 23893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:38.046537 23893 hybrid_clock.cc:648] HybridClock initialized: now 1781939378046506 us; error 43 us; skew 500 ppm
Jun 20 07:09:38 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:38.049048 23893 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:38.049875 23893 webserver.cc:460] Webserver started at http://127.8.24.2:34319/ using document root <none> and password file <none>
I20260620 07:09:38.050386 23893 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:38.050448 23893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:38.050676 23893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:38.052630 23893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "c4d35f5102734f5793671129e650bdb6"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "963538d6c15878d03178273c60a79ace"
server_key_iv: "63b98cff6f3dcb99abbca34585ecd39d"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.053108 23893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "c4d35f5102734f5793671129e650bdb6"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "963538d6c15878d03178273c60a79ace"
server_key_iv: "63b98cff6f3dcb99abbca34585ecd39d"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.056562 23893 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:38.058807 23908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.060011 23893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:38.060122 23893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "c4d35f5102734f5793671129e650bdb6"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "963538d6c15878d03178273c60a79ace"
server_key_iv: "63b98cff6f3dcb99abbca34585ecd39d"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.060220 23893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:38.070513 23893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:38.073052 23893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:38.073235 23893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:38.073765 23893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:38.074525 23893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:38.074569 23893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.074618 23893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:38.074646 23893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.082755 23893 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:32825
I20260620 07:09:38.082784 24021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:32825 every 8 connection(s)
I20260620 07:09:38.083617 23893 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
Jun 20 07:09:38 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:38.093055 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 23893
I20260620 07:09:38.093173 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:09:38.093502 8288 external_mini_cluster.cc:1428] Setting key bc1f12fceb7252fa1b520d164a8db0e4
I20260620 07:09:38.095865 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.085254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:42747 (local address 127.8.24.62:38805)
0620 07:09:38.085412 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.085417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.086244 (+ 827us) server_negotiation.cc:407] Connection header received
0620 07:09:38.087405 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.087407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.087476 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.087547 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:38.088585 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.089096 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.090112 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.090311 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.090374 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.092603 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:38.092606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:38.092608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:38.092633 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:38.094010 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.094493 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.094497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.094499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.094574 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.094870 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.094873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.094874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.095040 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:38.095121 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.095599 (+ 478us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.095715 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 07:09:38.096796 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.085683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:42747)
0620 07:09:38.086109 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.086148 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.087227 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.087692 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.087698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.088024 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:38.088481 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.088494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.089175 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.089177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.090029 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.090036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.090406 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.090408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.090544 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.091018 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:38.091033 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:38.092446 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:38.094112 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:38.094118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:38.094131 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:38.094403 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:38.094652 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:38.094656 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:38.094658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:38.094785 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:38.095133 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:38.095139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:38.095357 (+ 218us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.095545 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.095839 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":113,"threads_started":1}
I20260620 07:09:38.097785 24022 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:38.097996 24022 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:38.098423 24022 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:38.099380 23697 ts_manager.cc:194] Registered new tserver with Master: c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
I20260620 07:09:38.099941 23697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:42747
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:09:38.116673 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:38.222080 24027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:38.222327 24027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:38.222374 24027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:38.226356 24027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:38.226413 24027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:38.226482 24027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:38.231722 24027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:38.232681 24027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:38.233659 24027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:38.242487 24032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:38.242514 24033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:38.242544 24027 server_base.cc:1034] running on GCE node
W20260620 07:09:38.242487 24035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:38.243041 24027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:38.243610 24027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:38.244761 24027 hybrid_clock.cc:648] HybridClock initialized: now 1781939378244743 us; error 30 us; skew 500 ppm
Jun 20 07:09:38 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:38.247157 24027 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:38.247985 24027 webserver.cc:460] Webserver started at http://127.8.24.3:38119/ using document root <none> and password file <none>
I20260620 07:09:38.248482 24027 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:38.248543 24027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:38.248714 24027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:09:38.250667 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "9bad1af79bebcc8b3c38cdfa070b5ea4"
server_key_iv: "7e0da545dfe21ec06e8a311bd9131e64"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.251250 24027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "9bad1af79bebcc8b3c38cdfa070b5ea4"
server_key_iv: "7e0da545dfe21ec06e8a311bd9131e64"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.254712 24027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:09:38.257076 24042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.258378 24027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:09:38.258473 24027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "9bad1af79bebcc8b3c38cdfa070b5ea4"
server_key_iv: "7e0da545dfe21ec06e8a311bd9131e64"
server_key_version: "encryptionkey@0"
I20260620 07:09:38.258574 24027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:38.268395 24027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:38.270961 24027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:38.271099 24027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:38.271651 24027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:38.272434 24027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:38.272478 24027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.272516 24027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:38.272535 24027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:38.280313 24027 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46497
I20260620 07:09:38.280339 24155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46497 every 8 connection(s)
I20260620 07:09:38.281122 24027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:38.282064 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24027
I20260620 07:09:38.282156 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:09:38.282390 8288 external_mini_cluster.cc:1428] Setting key b18730ddb1c1e6a11612e7d02d21748e
Jun 20 07:09:38 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:38.293144 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.282777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:57149 (local address 127.8.24.62:38805)
0620 07:09:38.283035 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.283038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.283842 (+ 804us) server_negotiation.cc:407] Connection header received
0620 07:09:38.285121 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.285124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.285176 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.285245 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:38.286346 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.286858 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.287972 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.288189 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.288271 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.290320 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:38.290323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:38.290325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:38.290346 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:38.291369 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.291816 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.291820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.291821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.291889 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.292188 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.292191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.292193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.292350 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:38.292427 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.292871 (+ 444us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.292994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260620 07:09:38.293833 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.283223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:57149)
0620 07:09:38.283716 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.283764 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.284942 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.285410 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.285424 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.285744 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:38.286227 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.286240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.286964 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.286968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.287866 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.287876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.288310 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.288314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.288470 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.288909 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:38.288926 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:38.290180 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:38.291492 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:38.291497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:38.291510 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:38.291731 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:38.291973 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:38.291976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:38.291978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:38.292099 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:38.292446 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:38.292451 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:38.292658 (+ 207us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.292800 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.293079 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":143,"threads_started":1}
I20260620 07:09:38.294755 24156 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:38.294978 24156 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:38.295470 24156 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:38.296381 23697 ts_manager.cc:194] Registered new tserver with Master: 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497)
I20260620 07:09:38.296897 23697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:57149
I20260620 07:09:38.307312 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:38.315199 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.309097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.8.24.62:38805)
0620 07:09:38.309216 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.309221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.309366 (+ 145us) server_negotiation.cc:407] Connection header received
0620 07:09:38.309480 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.309482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.309521 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.309573 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:38.310140 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.310620 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.311371 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.311609 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.311680 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.312821 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:38.312823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:38.312825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:38.312846 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:38.313923 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.314254 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.314256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.314257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.314317 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:38.314492 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:38.314494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:38.314495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:38.314680 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:38.314771 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.314961 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.315057 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:09:38.317628 23697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:09:38.320534 23697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:09:38.333504 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.328270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.0.0.1:42752)
0620 07:09:38.328740 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.328778 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.328915 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.329422 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.329427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.329448 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:38.329584 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.329591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.330824 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.330827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.331766 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.331775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.332819 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.332823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.333059 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.333113 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.333227 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.333348 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":90,"thread_start_us":79,"threads_started":1}
I20260620 07:09:38.333515 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.328399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.8.24.1:36761)
0620 07:09:38.328718 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.328730 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.328867 (+ 137us) server_negotiation.cc:407] Connection header received
0620 07:09:38.329063 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.329068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.329293 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.329450 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:38.329687 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.330731 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.331903 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.332720 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.332823 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.332975 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.333258 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.333359 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
I20260620 07:09:38.334499 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.328975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:32825 (local address 127.0.0.1:34988)
0620 07:09:38.329214 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.329226 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.329300 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.330312 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.330315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.330334 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:38.330433 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.330439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.331871 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.331875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.333083 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.333092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.334064 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.334067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.334180 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.334203 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.334267 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.334332 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":87,"thread_start_us":101,"threads_started":1}
I20260620 07:09:38.334532 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.329131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.8.24.2:32825)
0620 07:09:38.329860 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.329872 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.329892 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:09:38.329975 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.329980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.330171 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.330316 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:38.330545 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.331746 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.333197 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.333959 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.334050 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.334181 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.334293 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.334373 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":102,"threads_started":1}
I20260620 07:09:38.334542 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.329282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46497 (local address 127.0.0.1:54916)
0620 07:09:38.329498 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.329510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.329590 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.330574 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.330577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.330589 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:38.330666 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.330670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.331935 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.331938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.333091 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.333100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.334131 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.334134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.334252 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.334275 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.334320 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.334384 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":92,"thread_start_us":72,"threads_started":1}
I20260620 07:09:38.335062 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.329739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.8.24.3:46497)
0620 07:09:38.330142 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.330153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.330172 (+ 19us) server_negotiation.cc:407] Connection header received
0620 07:09:38.330239 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.330244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.330430 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.330603 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:38.330747 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.331832 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.333236 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.333997 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.334671 (+ 674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.334799 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.334846 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.334933 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":112,"threads_started":1}
I20260620 07:09:38.336789 23822 tablet_service.cc:1467] Processing CreateTablet for tablet f82763618f4541eba5ba3da1cfd291a0 (DEFAULT_TABLE table=test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:38.337726 23822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82763618f4541eba5ba3da1cfd291a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:38.337519 24090 tablet_service.cc:1467] Processing CreateTablet for tablet f82763618f4541eba5ba3da1cfd291a0 (DEFAULT_TABLE table=test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:38.337518 23956 tablet_service.cc:1467] Processing CreateTablet for tablet f82763618f4541eba5ba3da1cfd291a0 (DEFAULT_TABLE table=test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:38.338532 23956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82763618f4541eba5ba3da1cfd291a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:38.338539 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82763618f4541eba5ba3da1cfd291a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:38.343403 24173 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap starting.
I20260620 07:09:38.345875 24175 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Bootstrap starting.
I20260620 07:09:38.346042 24173 tablet_bootstrap.cc:654] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:38.346710 24173 log.cc:824] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:38.347776 24174 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Bootstrap starting.
I20260620 07:09:38.347985 24175 tablet_bootstrap.cc:654] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:38.348475 24173 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: No bootstrap required, opened a new log
I20260620 07:09:38.348697 24173 ts_tablet_manager.cc:1397] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:09:38.348781 24175 log.cc:824] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:38.349699 24174 tablet_bootstrap.cc:654] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:38.350752 24175 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: No bootstrap required, opened a new log
I20260620 07:09:38.350826 24174 log.cc:824] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:38.350937 24175 ts_tablet_manager.cc:1397] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:09:38.351835 24173 raft_consensus.cc:348] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.352064 24173 raft_consensus.cc:374] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:38.352139 24173 raft_consensus.cc:729] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f831000e02fd4e2da033e2c6baf548ab, State: Initialized, Role: FOLLOWER
I20260620 07:09:38.352535 24173 consensus_queue.cc:260] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.352710 24174 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: No bootstrap required, opened a new log
I20260620 07:09:38.352900 24174 ts_tablet_manager.cc:1397] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:09:38.353482 24173 ts_tablet_manager.cc:1428] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:09:38.353869 23888 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
I20260620 07:09:38.354605 24175 raft_consensus.cc:348] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.354921 24175 raft_consensus.cc:374] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:38.354987 24175 raft_consensus.cc:729] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d35f5102734f5793671129e650bdb6, State: Initialized, Role: FOLLOWER
I20260620 07:09:38.355423 24175 consensus_queue.cc:260] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.356102 24022 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
I20260620 07:09:38.356452 24175 ts_tablet_manager.cc:1428] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260620 07:09:38.356570 24174 raft_consensus.cc:348] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.356840 24174 raft_consensus.cc:374] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:38.356901 24174 raft_consensus.cc:729] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b6d5746fa40b5885d9db64ac86d3f, State: Initialized, Role: FOLLOWER
I20260620 07:09:38.357298 24174 consensus_queue.cc:260] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.358043 24156 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
I20260620 07:09:38.358245 24174 ts_tablet_manager.cc:1428] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 07:09:38.390163 23889 tablet.cc:2369] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:38.419685 24181 raft_consensus.cc:482] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:38.419876 24181 raft_consensus.cc:504] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.420995 24181 leader_election.cc:288] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
I20260620 07:09:38.424831 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.421199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.8.24.3:41945)
0620 07:09:38.421328 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.421344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.421439 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.421806 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.421809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.421830 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:38.421952 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.421958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.422789 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.422792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.423750 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.423759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.424386 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.424389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.424488 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.424546 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.424608 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.424670 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":57}
I20260620 07:09:38.424844 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.421291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41945 (local address 127.8.24.1:36761)
0620 07:09:38.421423 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.421428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.421443 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:09:38.421563 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.421566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.421610 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.421799 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:38.422065 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.422687 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.423860 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.424278 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.424353 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.424403 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.424640 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.424711 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:09:38.424863 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.421454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:32825 (local address 127.8.24.3:38247)
0620 07:09:38.421837 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.421850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:38.421943 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:38.422158 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:38.422161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:38.422176 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:38.422288 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.422294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.422997 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.422999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.423834 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:38.423845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.424485 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.424487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:38.424578 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.424597 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:09:38.424667 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:09:38.424725 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":18,"thread_start_us":128,"threads_started":1}
I20260620 07:09:38.424863 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.421581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:38247 (local address 127.8.24.2:32825)
0620 07:09:38.421721 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.421725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.421908 (+ 183us) server_negotiation.cc:407] Connection header received
0620 07:09:38.422017 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.422020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.422069 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.422125 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:38.422379 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.422902 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.423955 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.424368 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.424475 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.424534 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.424690 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.424758 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 07:09:38.425504 23976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82763618f4541eba5ba3da1cfd291a0" candidate_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d35f5102734f5793671129e650bdb6" is_pre_election: true
I20260620 07:09:38.425505 23842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82763618f4541eba5ba3da1cfd291a0" candidate_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f831000e02fd4e2da033e2c6baf548ab" is_pre_election: true
I20260620 07:09:38.425808 23976 raft_consensus.cc:2454] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b6d5746fa40b5885d9db64ac86d3f in term 0.
I20260620 07:09:38.425808 23842 raft_consensus.cc:2454] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f9b6d5746fa40b5885d9db64ac86d3f in term 0.
I20260620 07:09:38.426259 24044 leader_election.cc:302] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b6d5746fa40b5885d9db64ac86d3f, f831000e02fd4e2da033e2c6baf548ab; no voters:
I20260620 07:09:38.426489 24181 raft_consensus.cc:2781] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:38.426565 24181 raft_consensus.cc:482] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:38.426615 24181 raft_consensus.cc:3037] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:38.427573 24181 raft_consensus.cc:504] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.427857 24181 leader_election.cc:288] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [CANDIDATE]: Term 1 election: Requested vote from peers f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
I20260620 07:09:38.428157 23842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82763618f4541eba5ba3da1cfd291a0" candidate_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f831000e02fd4e2da033e2c6baf548ab"
I20260620 07:09:38.428189 23976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82763618f4541eba5ba3da1cfd291a0" candidate_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d35f5102734f5793671129e650bdb6"
I20260620 07:09:38.428283 23976 raft_consensus.cc:3037] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:38.428279 23842 raft_consensus.cc:3037] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:38.429471 23976 raft_consensus.cc:2454] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b6d5746fa40b5885d9db64ac86d3f in term 1.
I20260620 07:09:38.429471 23842 raft_consensus.cc:2454] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f9b6d5746fa40b5885d9db64ac86d3f in term 1.
I20260620 07:09:38.429733 24044 leader_election.cc:302] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b6d5746fa40b5885d9db64ac86d3f, f831000e02fd4e2da033e2c6baf548ab; no voters:
I20260620 07:09:38.429874 24181 raft_consensus.cc:2781] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:38.430076 24181 raft_consensus.cc:686] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 LEADER]: Becoming Leader. State: Replica: 5f9b6d5746fa40b5885d9db64ac86d3f, State: Running, Role: LEADER
I20260620 07:09:38.430356 24181 consensus_queue.cc:237] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:38.433300 23695 catalog_manager.cc:5495] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3). New cstate: current_term: 1 leader_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } health_report { overall_health: HEALTHY } } }
I20260620 07:09:38.462950 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.459568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.8.24.3:46497)
0620 07:09:38.459674 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:38.459677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:38.459690 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:09:38.459819 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:38.459821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:38.459879 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:38.459971 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:38.460145 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.460628 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.461412 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:38.461613 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:38.461737 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:38.462100 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:38.462566 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:38.462657 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:38.462713 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 07:09:38.462793 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:09:38.469265 23976 raft_consensus.cc:1261] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Refusing update from remote peer 5f9b6d5746fa40b5885d9db64ac86d3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:38.469961 24183 consensus_queue.cc:1035] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:38.470067 23842 raft_consensus.cc:1261] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Refusing update from remote peer 5f9b6d5746fa40b5885d9db64ac86d3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:38.470547 24183 consensus_queue.cc:1035] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:38.479745 24187 mvcc.cc:205] Tried to move back new op lower bound from 7298823694201933824 to 7298823694053351424. Current Snapshot: MvccSnapshot[applied={T|T < 7298823694201933824}]
I20260620 07:09:38.485951 23697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:38.486131 23697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:38.488406 23697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Sending DeleteTablet for 3 replicas of tablet f82763618f4541eba5ba3da1cfd291a0
I20260620 07:09:38.489090 23822 tablet_service.cc:1514] Processing DeleteTablet for tablet f82763618f4541eba5ba3da1cfd291a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:38 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:42752
I20260620 07:09:38.489166 23956 tablet_service.cc:1514] Processing DeleteTablet for tablet f82763618f4541eba5ba3da1cfd291a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:38 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:34988
I20260620 07:09:38.489531 24196 tablet_replica.cc:330] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: stopping tablet replica
I20260620 07:09:38.489712 24090 tablet_service.cc:1514] Processing DeleteTablet for tablet f82763618f4541eba5ba3da1cfd291a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:38 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:54916
I20260620 07:09:38.489797 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23759
I20260620 07:09:38.489878 24196 raft_consensus.cc:2229] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 07:09:38.495260 23682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:36761 (error 108)
I20260620 07:09:38.495471 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23893
I20260620 07:09:38.495489 24198 tablet_replica.cc:330] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: stopping tablet replica
I20260620 07:09:38.495757 24198 raft_consensus.cc:2229] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:09:38.496084 24198 raft_consensus.cc:2258] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:38.497818 24198 ts_tablet_manager.cc:1905] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:38.498123 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.497864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.0.0.1:42756)
0620 07:09:38.497989 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.498054 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 07:09:38.498275 23682 catalog_manager.cc:4584] TS f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f82763618f4541eba5ba3da1cfd291a0: Network error: Client connection negotiation failed: client connection to 127.8.24.1:36761: connect: Connection refused (error 111)
I20260620 07:09:38.500489 24198 ts_tablet_manager.cc:1918] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:38.500571 24198 log.cc:1196] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/f82763618f4541eba5ba3da1cfd291a0
I20260620 07:09:38.500859 24198 ts_tablet_manager.cc:1939] T f82763618f4541eba5ba3da1cfd291a0 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting consensus metadata
I20260620 07:09:38.501965 23682 catalog_manager.cc:4841] TS 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): tablet f82763618f4541eba5ba3da1cfd291a0 (table test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]) successfully deleted
I20260620 07:09:38.502066 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24027
I20260620 07:09:38.502460 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:38.502235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:32825 (local address 127.0.0.1:35004)
0620 07:09:38.502360 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:38.502398 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:32825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 07:09:38.506718 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 23665
I20260620 07:09:58.511942 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--webserver_interface=127.8.24.62
--webserver_port=45449
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:09:58.617377 24200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:58.617625 24200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:58.617666 24200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:58.621721 24200 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 07:09:58.621783 24200 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 07:09:58.621798 24200 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 07:09:58.621809 24200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:09:58.621826 24200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:58.621861 24200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:58.621888 24200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:09:58.621913 24200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:09:58.627174 24200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:58.628242 24200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:58.629253 24200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:09:58.642133 24200 server_base.cc:1034] running on GCE node
W20260620 07:09:58.642079 24206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:58.642156 24209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:58.642079 24207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:58.642920 24200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:58.644129 24200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:58.645289 24200 hybrid_clock.cc:648] HybridClock initialized: now 1781939398645270 us; error 43 us; skew 500 ppm
Jun 20 07:09:58 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:58.648095 24200 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:09:58.649013 24200 webserver.cc:460] Webserver started at http://127.8.24.62:45449/ using document root <none> and password file <none>
I20260620 07:09:58.649567 24200 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:58.649639 24200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:58.653532 24200 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:09:58.655637 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:58.656858 24200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:58.656976 24200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ae76e7ce9b7f4b58b56f92b907238a5c"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "17dd728ecc187cddc37684722b64ad81"
server_key_iv: "abcde1b99f5070621e35d9e9a676fc5e"
server_key_version: "encryptionkey@0"
I20260620 07:09:58.657599 24200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:58.678088 24200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:58.680743 24200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:58.680912 24200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:58.687512 24200 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38805
I20260620 07:09:58.687784 24268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38805 every 8 connection(s)
I20260620 07:09:58.688561 24200 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:09:58.692674 24269 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:09:58.694199 24269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap starting.
I20260620 07:09:58.697887 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24200
I20260620 07:09:58.698274 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:36761
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=38165
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:58.710870 24269 log.cc:824] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:58.719463 24269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:58.719801 24269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap complete.
I20260620 07:09:58.723899 24269 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:58.724323 24269 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Initialized, Role: FOLLOWER
I20260620 07:09:58.724795 24269 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:58.724929 24269 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:09:58.725000 24269 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:09:58.725097 24269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 1 FOLLOWER]: Advancing to term 2
I20260620 07:09:58.726948 24269 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:58.727262 24269 leader_election.cc:302] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae76e7ce9b7f4b58b56f92b907238a5c; no voters:
I20260620 07:09:58.727579 24269 leader_election.cc:288] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 07:09:58.727666 24273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Leader election won for term 2
I20260620 07:09:58.727860 24273 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 LEADER]: Becoming Leader. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Running, Role: LEADER
I20260620 07:09:58.728216 24273 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:09:58.728819 24269 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:09:58.729609 24275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae76e7ce9b7f4b58b56f92b907238a5c. Latest consensus state: current_term: 2 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:09:58.729727 24275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:58.729974 24274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:09:58.730044 24274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:09:58.730059 24282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:09:58.734648 24282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:09:58.735266 24282 catalog_manager.cc:1259] Loaded cluster ID: 9301fe9fb3cf4226847b097871078432
I20260620 07:09:58.735316 24282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:09:58.736984 24282 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:09:58.737555 24282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Loaded TSK: 0
I20260620 07:09:58.747586 24282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Generated new TSK 1
I20260620 07:09:58.747816 24282 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:09:58.817251 24271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:58.817497 24271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:58.817554 24271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:58.821512 24271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:58.821568 24271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:58.821641 24271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:09:58.826696 24271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:36761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=38165
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:58.827656 24271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:58.828594 24271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:58.837836 24297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:58.837814 24296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:58.838207 24299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:58.838440 24271 server_base.cc:1034] running on GCE node
I20260620 07:09:58.838790 24271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:58.839375 24271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:58.840566 24271 hybrid_clock.cc:648] HybridClock initialized: now 1781939398840555 us; error 27 us; skew 500 ppm
Jun 20 07:09:58 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:58.843036 24271 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:09:58.843923 24271 webserver.cc:460] Webserver started at http://127.8.24.1:38165/ using document root <none> and password file <none>
I20260620 07:09:58.844434 24271 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:58.844501 24271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:58.848250 24271 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:09:58.850361 24306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:58.851594 24271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:09:58.851686 24271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "f831000e02fd4e2da033e2c6baf548ab"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "d76744b8ba4ee9655fe261046cdb8a7b"
server_key_iv: "064f1160d239e124e99376bfdeea632a"
server_key_version: "encryptionkey@0"
I20260620 07:09:58.852212 24271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:58.862550 24271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:58.865615 24271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:58.865782 24271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:58.866381 24271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:58.867480 24313 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 07:09:58.871461 24271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 07:09:58.871520 24271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 07:09:58.871570 24271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 07:09:58.873167 24271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 07:09:58.873221 24271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 07:09:58.873508 24313 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap starting.
I20260620 07:09:58.883064 24271 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:36761
I20260620 07:09:58.883108 24420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:36761 every 8 connection(s)
I20260620 07:09:58.884219 24271 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:09:58.893486 24313 log.cc:824] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:58.894367 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24271
I20260620 07:09:58.894833 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:32825
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=34319
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:09:58 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:58.901405 24313 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:58.901878 24313 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap complete.
I20260620 07:09:58.902073 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:58.886382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:34667)
0620 07:09:58.886850 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:58.886893 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:58.887669 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:58.888955 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:58.888964 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:58.889422 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:58.890127 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:58.890150 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.891524 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.891527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:58.892771 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:58.892781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.893485 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.893489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:58.893666 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:58.894353 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:58.894372 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:58.896084 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:58.898541 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:58.898547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:58.898562 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:58.898895 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:58.899309 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:58.899312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:58.899314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:58.899509 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:58.900170 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:58.900176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:58.900485 (+ 309us) client_negotiation.cc:769] Sending connection context
0620 07:09:58.900683 (+ 198us) client_negotiation.cc:240] Negotiation successful
0620 07:09:58.901023 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260620 07:09:58.902061 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:58.886267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34667 (local address 127.8.24.62:38805)
0620 07:09:58.888047 (+ 1780us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:58.888061 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:58.888091 (+ 30us) server_negotiation.cc:407] Connection header received
0620 07:09:58.888297 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:58.888307 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:58.888573 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:58.888812 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:58.890334 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.891331 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.892881 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.893349 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.893467 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:58.896296 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:58.896302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:58.896318 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:58.896351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:58.898332 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:58.899033 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:58.899039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:58.899044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:58.899210 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:58.899614 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:58.899617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:58.899619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:58.900041 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:58.900176 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:58.900845 (+ 669us) server_negotiation.cc:299] Negotiation successful
0620 07:09:58.901154 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1664,"thread_start_us":89,"threads_started":1}
I20260620 07:09:58.903016 24313 ts_tablet_manager.cc:1397] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.004s
I20260620 07:09:58.905066 24421 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:58.905295 24421 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:58.905797 24421 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:58.905606 24313 raft_consensus.cc:348] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:58.906032 24313 raft_consensus.cc:729] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f831000e02fd4e2da033e2c6baf548ab, State: Initialized, Role: FOLLOWER
I20260620 07:09:58.906469 24313 consensus_queue.cc:260] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:58.907397 24313 ts_tablet_manager.cc:1428] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260620 07:09:58.908980 24233 ts_manager.cc:194] Registered new tserver with Master: f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761)
I20260620 07:09:58.912170 24233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:34667
I20260620 07:09:58.914605 24421 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
Jun 20 07:09:58 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.1@KRBTEST.COM
I20260620 07:09:58.921530 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:58.911555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.0.0.1:60706)
0620 07:09:58.911947 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:58.911990 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:58.912103 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:58.912711 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:58.912718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:58.913126 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:58.913253 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:58.913261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.914550 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.914554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:58.915502 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:58.915513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.916202 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.916218 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:58.916328 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:58.916814 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:58.916828 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:58.918241 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:58.919957 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:58.919965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:58.919971 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:58.920183 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:58.920451 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:58.920454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:58.920456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:58.920533 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:58.920940 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:58.920946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:58.921050 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 07:09:58.921170 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 07:09:58.921343 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260620 07:09:58.921547 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:58.911634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.8.24.1:36761)
0620 07:09:58.912265 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:58.912277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:58.912299 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:09:58.912359 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:58.912364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:58.912547 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:58.912687 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:58.913374 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.914434 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.915619 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:58.916093 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:58.916194 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:58.918364 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:58.918372 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:58.918381 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:58.918401 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:58.919852 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:58.920289 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:58.920294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:58.920295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:58.920365 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:58.920617 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:58.920620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:58.920622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:58.920829 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:58.920941 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:58.921264 (+ 323us) server_negotiation.cc:299] Negotiation successful
0620 07:09:58.921407 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":541,"thread_start_us":96,"threads_started":1}
I20260620 07:09:58.922444 24355 tablet_service.cc:1514] Processing DeleteTablet for tablet f82763618f4541eba5ba3da1cfd291a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:38 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:60706
I20260620 07:09:58.923296 24430 tablet_replica.cc:330] stopping tablet replica
I20260620 07:09:58.923589 24430 raft_consensus.cc:2229] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:58.923801 24430 raft_consensus.cc:2258] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:58.924705 24430 ts_tablet_manager.cc:1905] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:58.927973 24430 ts_tablet_manager.cc:1918] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:58.928054 24430 log.cc:1196] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal/wals/f82763618f4541eba5ba3da1cfd291a0
I20260620 07:09:58.928282 24430 ts_tablet_manager.cc:1939] T f82763618f4541eba5ba3da1cfd291a0 P f831000e02fd4e2da033e2c6baf548ab: Deleting consensus metadata
I20260620 07:09:58.929724 24218 catalog_manager.cc:4841] TS f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761): tablet f82763618f4541eba5ba3da1cfd291a0 (table test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]) successfully deleted
W20260620 07:09:59.005008 24426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:59.005271 24426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:59.005316 24426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:59.009428 24426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:59.009521 24426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:59.009603 24426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:09:59.014930 24426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:32825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=34319
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:59.016036 24426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:59.017057 24426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:59.026472 24438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:59.026472 24435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:59.026584 24426 server_base.cc:1034] running on GCE node
W20260620 07:09:59.026542 24436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:59.027107 24426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:59.027755 24426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:59.028899 24426 hybrid_clock.cc:648] HybridClock initialized: now 1781939399028883 us; error 29 us; skew 500 ppm
Jun 20 07:09:59 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939399, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:59.031540 24426 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:09:59.032438 24426 webserver.cc:460] Webserver started at http://127.8.24.2:34319/ using document root <none> and password file <none>
I20260620 07:09:59.032963 24426 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:59.033035 24426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:59.037015 24426 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 07:09:59.039052 24445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:59.040259 24426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:59.040361 24426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "c4d35f5102734f5793671129e650bdb6"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "963538d6c15878d03178273c60a79ace"
server_key_iv: "63b98cff6f3dcb99abbca34585ecd39d"
server_key_version: "encryptionkey@0"
I20260620 07:09:59.040875 24426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:59.058342 24426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:59.061308 24426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:59.061479 24426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:59.062058 24426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:59.063086 24452 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 07:09:59.067124 24426 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 07:09:59.067188 24426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 07:09:59.067262 24426 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 07:09:59.068953 24426 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 07:09:59.068998 24426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 07:09:59.069293 24452 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Bootstrap starting.
I20260620 07:09:59.078545 24426 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:32825
I20260620 07:09:59.078625 24559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:32825 every 8 connection(s)
I20260620 07:09:59.079504 24426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:09:59.081318 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24426
I20260620 07:09:59.081801 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:46497
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=38119
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:09:59.087505 24452 log.cc:824] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Log is configured to *not* fsync() on all Append() calls
Jun 20 07:09:59 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939399, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:59.095793 24452 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:09:59.096215 24452 tablet_bootstrap.cc:492] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Bootstrap complete.
I20260620 07:09:59.097560 24452 ts_tablet_manager.cc:1397] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Time spent bootstrapping tablet: real 0.028s user 0.020s sys 0.008s
I20260620 07:09:59.099004 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.081209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51173 (local address 127.8.24.62:38805)
0620 07:09:59.081366 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.081372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.081998 (+ 626us) server_negotiation.cc:407] Connection header received
0620 07:09:59.083015 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.083019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.083079 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.083152 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:59.084694 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.085528 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.086677 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.086911 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.086999 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.089791 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:59.089795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:59.089798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:59.089829 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:59.096661 (+ 6832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.097264 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.097268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.097270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.097356 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.097743 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.097747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.097748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.097980 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:59.098053 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.098673 (+ 620us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.098830 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:09:59.100585 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.081363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:51173)
0620 07:09:59.081851 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.081903 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.082807 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.083339 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.083349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.083851 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:59.084498 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.084538 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.085673 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.085675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.086555 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.086565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.087032 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.087036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.087200 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.087790 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:59.087806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:59.089594 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:59.096815 (+ 7221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.096821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.096835 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.097150 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.097469 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.097471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.097473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.097628 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.098127 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:59.098133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:59.098416 (+ 283us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.099305 (+ 889us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.099655 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260620 07:09:59.100484 24452 raft_consensus.cc:348] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:59.100991 24452 raft_consensus.cc:729] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d35f5102734f5793671129e650bdb6, State: Initialized, Role: FOLLOWER
I20260620 07:09:59.101480 24452 consensus_queue.cc:260] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } } peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } }
I20260620 07:09:59.101962 24560 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:59.102167 24560 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:59.102494 24452 ts_tablet_manager.cc:1428] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:09:59.102674 24560 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:59.104262 24233 ts_manager.cc:194] Registered new tserver with Master: c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
I20260620 07:09:59.105479 24233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:51173
I20260620 07:09:59.107657 24560 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
Jun 20 07:09:59 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.2@KRBTEST.COM
I20260620 07:09:59.114380 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.105145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:32825 (local address 127.0.0.1:58010)
0620 07:09:59.105675 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.105687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.105768 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.106271 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.106274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.106453 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:59.106550 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.106557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.107879 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.107883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.108731 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.108742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.109252 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.109267 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.109359 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.109818 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:59.109831 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:59.111182 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:59.112857 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.112861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.112863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.113121 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.113383 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.113387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.113389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.113482 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.113875 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:59.113878 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:59.113978 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.114079 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.114221 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461}
I20260620 07:09:59.114403 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.105254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.8.24.2:32825)
0620 07:09:59.105522 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.105530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.105858 (+ 328us) server_negotiation.cc:407] Connection header received
0620 07:09:59.105920 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.105924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.106124 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.106261 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:59.106678 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.107751 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.108835 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.109149 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.109251 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.111307 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:59.111315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:59.111324 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:59.111344 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:59.112749 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.113215 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.113219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.113220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.113302 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.113562 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.113564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.113565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.113778 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:59.113873 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.114152 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.114289 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260620 07:09:59.115377 24494 tablet_service.cc:1514] Processing DeleteTablet for tablet f82763618f4541eba5ba3da1cfd291a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:38 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:58010
I20260620 07:09:59.116240 24567 tablet_replica.cc:330] stopping tablet replica
I20260620 07:09:59.116549 24567 raft_consensus.cc:2229] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:59.116809 24567 raft_consensus.cc:2258] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:59.117887 24567 ts_tablet_manager.cc:1905] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:59.122345 24567 ts_tablet_manager.cc:1918] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:59.122444 24567 log.cc:1196] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/f82763618f4541eba5ba3da1cfd291a0
I20260620 07:09:59.122709 24567 ts_tablet_manager.cc:1939] T f82763618f4541eba5ba3da1cfd291a0 P c4d35f5102734f5793671129e650bdb6: Deleting consensus metadata
I20260620 07:09:59.124552 24218 catalog_manager.cc:4841] TS c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): tablet f82763618f4541eba5ba3da1cfd291a0 (table test-table [id=6c569bc211d54f20b7cd1bb99b1f092e]) successfully deleted
W20260620 07:09:59.201510 24563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:09:59.201781 24563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:09:59.201823 24563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:09:59.205812 24563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:09:59.205875 24563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:09:59.205931 24563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:09:59.211067 24563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:46497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=38119
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:09:59.212078 24563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:09:59.213032 24563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:09:59.221979 24575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:59.221980 24573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:09:59.221992 24572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:09:59.222026 24563 server_base.cc:1034] running on GCE node
I20260620 07:09:59.222674 24563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:09:59.223250 24563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:09:59.224395 24563 hybrid_clock.cc:648] HybridClock initialized: now 1781939399224365 us; error 43 us; skew 500 ppm
Jun 20 07:09:59 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939399, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:09:59.226855 24563 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:09:59.227677 24563 webserver.cc:460] Webserver started at http://127.8.24.3:38119/ using document root <none> and password file <none>
I20260620 07:09:59.228184 24563 fs_manager.cc:359] Metadata directory not provided
I20260620 07:09:59.228267 24563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:09:59.232154 24563 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 07:09:59.234117 24582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:59.235388 24563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:59.235508 24563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "9bad1af79bebcc8b3c38cdfa070b5ea4"
server_key_iv: "7e0da545dfe21ec06e8a311bd9131e64"
server_key_version: "encryptionkey@0"
I20260620 07:09:59.236029 24563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:09:59.282562 24563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:09:59.285401 24563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:09:59.285585 24563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:09:59.286173 24563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:09:59.286975 24563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:09:59.287019 24563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:59.287071 24563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:09:59.287108 24563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:09:59.295265 24563 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46497
I20260620 07:09:59.295300 24695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46497 every 8 connection(s)
I20260620 07:09:59.296208 24563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:09:59.300154 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24563
Jun 20 07:09:59 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939399, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:09:59.307859 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.300739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.8.24.62:38805)
0620 07:09:59.300943 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.300947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.301103 (+ 156us) server_negotiation.cc:407] Connection header received
0620 07:09:59.301236 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.301239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.301275 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.301343 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:59.302013 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.302530 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.303356 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.303612 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.303705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.305033 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:59.305036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:59.305038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:59.305058 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:59.306342 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.306719 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.306723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.306724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.306795 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.307036 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.307039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.307041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.307212 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:59.307393 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.307553 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.307665 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
I20260620 07:09:59.309310 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.297889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:60453 (local address 127.8.24.62:38805)
0620 07:09:59.298009 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.298013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.298822 (+ 809us) server_negotiation.cc:407] Connection header received
0620 07:09:59.300146 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.300149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.300202 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.300281 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:59.301634 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.302138 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.303227 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.303459 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.303545 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.305914 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:59.305917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:59.305919 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:59.305952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:59.307143 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.307748 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.307751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.307752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.307836 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.308205 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.308208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.308210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.308400 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:59.308475 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.309035 (+ 560us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.309174 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:09:59.310019 24698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.298256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:60453)
0620 07:09:59.298678 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.298721 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.299899 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.300547 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.300556 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.301019 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:09:59.301486 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.301501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.302254 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.302257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.303112 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.303121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.303598 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.303602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.303760 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.304284 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:09:59.304304 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:09:59.305754 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:09:59.307247 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.307252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.307266 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.307604 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.307933 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:09:59.307935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:09:59.307938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:09:59.308104 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:09:59.308502 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:09:59.308508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:09:59.308787 (+ 279us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.308971 (+ 184us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.309212 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":111,"threads_started":1}
I20260620 07:09:59.310962 24696 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38805
I20260620 07:09:59.311154 24696 heartbeater.cc:464] Registering TS with master...
I20260620 07:09:59.311601 24696 heartbeater.cc:511] Master 127.8.24.62:38805 requested a full tablet report, sending...
I20260620 07:09:59.312494 24233 ts_manager.cc:194] Registered new tserver with Master: 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497)
I20260620 07:09:59.312986 24233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:60453
I20260620 07:09:59.320950 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:09:59.324302 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.321474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.8.24.62:38805)
0620 07:09:59.321574 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.321577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.321750 (+ 173us) server_negotiation.cc:407] Connection header received
0620 07:09:59.321862 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.321864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.321950 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.322024 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:59.322190 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.322656 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.323383 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.323631 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.323703 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.323926 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:59.324040 (+ 114us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 07:09:59.324213 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45452: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 07:09:59.324352 24423 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45452: token expired
W20260620 07:09:59.324659 24159 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.24.62:38805)
I20260620 07:09:59.330518 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.324967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.8.24.62:38805)
0620 07:09:59.325061 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.325064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.325075 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:09:59.325144 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.325145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.325176 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.325240 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:09:59.325729 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.326164 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.326837 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.327015 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.327077 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.328196 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:09:59.328199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:09:59.328201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:09:59.328221 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:09:59.329318 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.329648 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.329650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.329652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.329712 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:09:59.329889 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:09:59.329891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:09:59.329892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:09:59.330041 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:09:59.330142 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.330336 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.330415 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:09:59.332659 24233 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:09:59.335755 24233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:09:59.342672 24494 tablet_service.cc:1467] Processing CreateTablet for tablet 304e72b0c8c04c66924df2130b393543 (DEFAULT_TABLE table=test-table [id=853bc2048f7346ac92f49059469f65c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:59.342672 24355 tablet_service.cc:1467] Processing CreateTablet for tablet 304e72b0c8c04c66924df2130b393543 (DEFAULT_TABLE table=test-table [id=853bc2048f7346ac92f49059469f65c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:59.343279 24355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304e72b0c8c04c66924df2130b393543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:59.343278 24494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304e72b0c8c04c66924df2130b393543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:59.346022 24452 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Bootstrap starting.
I20260620 07:09:59.346143 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.341840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46497 (local address 127.0.0.1:34100)
0620 07:09:59.341944 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.341958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.342036 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.342769 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.342772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.342789 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:59.342881 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.342886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.344102 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.344105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.344929 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.344937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.345741 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.345744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.345826 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.345864 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.345933 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.346002 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 07:09:59.346791 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.341993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.8.24.3:46497)
0620 07:09:59.342355 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.342367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.342387 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:09:59.342449 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.342453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.342623 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.342765 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:59.342971 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.343977 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.345050 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.345628 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.345719 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.345864 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.346389 (+ 525us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.346647 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":160,"threads_started":1}
I20260620 07:09:59.346931 24452 tablet_bootstrap.cc:654] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:59.347592 24313 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap starting.
I20260620 07:09:59.348281 24313 tablet_bootstrap.cc:654] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:59.348294 24452 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: No bootstrap required, opened a new log
I20260620 07:09:59.348364 24452 ts_tablet_manager.cc:1397] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 07:09:59.348747 24452 raft_consensus.cc:348] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.348846 24452 raft_consensus.cc:374] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:59.348878 24452 raft_consensus.cc:729] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d35f5102734f5793671129e650bdb6, State: Initialized, Role: FOLLOWER
I20260620 07:09:59.348992 24452 consensus_queue.cc:260] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.349646 24452 ts_tablet_manager.cc:1428] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:09:59.349763 24630 tablet_service.cc:1467] Processing CreateTablet for tablet 304e72b0c8c04c66924df2130b393543 (DEFAULT_TABLE table=test-table [id=853bc2048f7346ac92f49059469f65c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:09:59.350859 24630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304e72b0c8c04c66924df2130b393543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:09:59.351200 24313 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: No bootstrap required, opened a new log
I20260620 07:09:59.351291 24313 ts_tablet_manager.cc:1397] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 07:09:59.351708 24313 raft_consensus.cc:348] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.351790 24313 raft_consensus.cc:374] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:59.351814 24313 raft_consensus.cc:729] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f831000e02fd4e2da033e2c6baf548ab, State: Initialized, Role: FOLLOWER
I20260620 07:09:59.351922 24313 consensus_queue.cc:260] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.352172 24313 ts_tablet_manager.cc:1428] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 07:09:59.357729 24708 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Bootstrap starting.
I20260620 07:09:59.359709 24708 tablet_bootstrap.cc:654] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Neither blocks nor log segments found. Creating new log.
I20260620 07:09:59.360707 24708 log.cc:824] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Log is configured to *not* fsync() on all Append() calls
I20260620 07:09:59.362461 24708 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: No bootstrap required, opened a new log
I20260620 07:09:59.362638 24708 ts_tablet_manager.cc:1397] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:09:59.365635 24708 raft_consensus.cc:348] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.365880 24708 raft_consensus.cc:374] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:09:59.365942 24708 raft_consensus.cc:729] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9b6d5746fa40b5885d9db64ac86d3f, State: Initialized, Role: FOLLOWER
I20260620 07:09:59.366348 24708 consensus_queue.cc:260] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.366962 24696 heartbeater.cc:503] Master 127.8.24.62:38805 was elected leader, sending a full tablet report...
I20260620 07:09:59.367288 24708 ts_tablet_manager.cc:1428] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 07:09:59.385392 24422 tablet.cc:2369] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:09:59.385947 24565 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:09:59.386117 24565 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.387146 24565 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761)
I20260620 07:09:59.391369 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.387391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46497 (local address 127.8.24.2:44081)
0620 07:09:59.387523 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.387538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.387644 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.388006 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.388009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.388030 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:59.388141 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.388147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.389056 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.389059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.390230 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.390240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.391008 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.391011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.391091 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.391129 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.391188 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.391245 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 07:09:59.391413 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.387501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:44081 (local address 127.8.24.3:46497)
0620 07:09:59.387615 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.387618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.387632 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:09:59.387808 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.387812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.387866 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.387945 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:59.388256 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.388954 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.390362 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.390897 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.390996 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.391057 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.391223 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.391299 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:09:59.391790 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.387809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:40057 (local address 127.8.24.1:36761)
0620 07:09:59.387953 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.387958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.388136 (+ 178us) server_negotiation.cc:407] Connection header received
0620 07:09:59.388217 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.388220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.388277 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.388347 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:09:59.388625 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.389326 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.390613 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.391211 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.391307 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.391484 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.391602 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.391683 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 07:09:59.392036 24650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304e72b0c8c04c66924df2130b393543" candidate_uuid: "c4d35f5102734f5793671129e650bdb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" is_pre_election: true
I20260620 07:09:59.392122 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.387699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.8.24.2:40057)
0620 07:09:59.388050 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.388063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:09:59.388144 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:09:59.388379 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:09:59.388381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:09:59.388398 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:09:59.388520 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.388526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.389424 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.389427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.390497 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:09:59.390503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.391340 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.391343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:09:59.391451 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.391474 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:09:59.391922 (+ 448us) client_negotiation.cc:240] Negotiation successful
0620 07:09:59.391993 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260620 07:09:59.392354 24650 raft_consensus.cc:2454] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d35f5102734f5793671129e650bdb6 in term 0.
I20260620 07:09:59.392684 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304e72b0c8c04c66924df2130b393543" candidate_uuid: "c4d35f5102734f5793671129e650bdb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f831000e02fd4e2da033e2c6baf548ab" is_pre_election: true
I20260620 07:09:59.392803 24447 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6; no voters:
I20260620 07:09:59.392957 24373 raft_consensus.cc:2454] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d35f5102734f5793671129e650bdb6 in term 0.
I20260620 07:09:59.393102 24565 raft_consensus.cc:2781] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:09:59.393204 24565 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:09:59.393261 24565 raft_consensus.cc:3037] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:59.394274 24565 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.394594 24565 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [CANDIDATE]: Term 1 election: Requested vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761)
I20260620 07:09:59.394904 24650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304e72b0c8c04c66924df2130b393543" candidate_uuid: "c4d35f5102734f5793671129e650bdb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
I20260620 07:09:59.394934 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304e72b0c8c04c66924df2130b393543" candidate_uuid: "c4d35f5102734f5793671129e650bdb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f831000e02fd4e2da033e2c6baf548ab"
I20260620 07:09:59.394999 24650 raft_consensus.cc:3037] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:59.395043 24373 raft_consensus.cc:3037] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:09:59.396070 24650 raft_consensus.cc:2454] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d35f5102734f5793671129e650bdb6 in term 1.
I20260620 07:09:59.396118 24373 raft_consensus.cc:2454] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d35f5102734f5793671129e650bdb6 in term 1.
I20260620 07:09:59.396353 24447 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6; no voters:
I20260620 07:09:59.396513 24565 raft_consensus.cc:2781] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:09:59.396744 24565 raft_consensus.cc:686] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 1 LEADER]: Becoming Leader. State: Replica: c4d35f5102734f5793671129e650bdb6, State: Running, Role: LEADER
I20260620 07:09:59.396991 24565 consensus_queue.cc:237] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:09:59.399756 24232 catalog_manager.cc:5495] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d35f5102734f5793671129e650bdb6 (127.8.24.2). New cstate: current_term: 1 leader_uuid: "c4d35f5102734f5793671129e650bdb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } health_report { overall_health: HEALTHY } } }
I20260620 07:09:59.424881 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.421869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.8.24.2:32825)
0620 07:09:59.421990 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:09:59.421994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:09:59.422007 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:09:59.422078 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:09:59.422081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:09:59.422130 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:09:59.422212 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:09:59.422392 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.422904 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.423688 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:09:59.423924 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:09:59.424000 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:09:59.424174 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:09:59.424572 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:09:59.424651 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 07:09:59.424694 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 07:09:59.424749 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:09:59.428434 24373 raft_consensus.cc:1261] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Refusing update from remote peer c4d35f5102734f5793671129e650bdb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:59.428431 24650 raft_consensus.cc:1261] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Refusing update from remote peer c4d35f5102734f5793671129e650bdb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:09:59.429042 24712 consensus_queue.cc:1035] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:59.429248 24565 consensus_queue.cc:1035] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:09:59.444335 24233 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:59.444476 24233 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:09:59.446596 24233 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Sending DeleteTablet for 3 replicas of tablet 304e72b0c8c04c66924df2130b393543
I20260620 07:09:59.447201 24630 tablet_service.cc:1514] Processing DeleteTablet for tablet 304e72b0c8c04c66924df2130b393543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:59 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:34100
I20260620 07:09:59.447257 24355 tablet_service.cc:1514] Processing DeleteTablet for tablet 304e72b0c8c04c66924df2130b393543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:59 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:60706
I20260620 07:09:59.447680 24726 tablet_replica.cc:330] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: stopping tablet replica
I20260620 07:09:59.447716 24494 tablet_service.cc:1514] Processing DeleteTablet for tablet 304e72b0c8c04c66924df2130b393543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:09:59 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:58010
I20260620 07:09:59.447800 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24271
I20260620 07:09:59.447882 24567 tablet_replica.cc:330] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: stopping tablet replica
I20260620 07:09:59.447983 24567 raft_consensus.cc:2229] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:09:59.448268 24567 raft_consensus.cc:2258] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:59.448417 24726 raft_consensus.cc:2229] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:09:59.448691 24726 raft_consensus.cc:2258] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:09:59.449225 24567 ts_tablet_manager.cc:1905] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:59.450119 24726 ts_tablet_manager.cc:1905] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:09:59.451730 24567 ts_tablet_manager.cc:1918] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:59.451797 24567 log.cc:1196] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/304e72b0c8c04c66924df2130b393543
I20260620 07:09:59.452021 24567 ts_tablet_manager.cc:1939] T 304e72b0c8c04c66924df2130b393543 P c4d35f5102734f5793671129e650bdb6: Deleting consensus metadata
I20260620 07:09:59.452550 24726 ts_tablet_manager.cc:1918] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:09:59.452703 24726 log.cc:1196] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/304e72b0c8c04c66924df2130b393543
I20260620 07:09:59.452790 24218 catalog_manager.cc:4841] TS c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): tablet 304e72b0c8c04c66924df2130b393543 (table test-table [id=853bc2048f7346ac92f49059469f65c2]) successfully deleted
I20260620 07:09:59.453091 24726 ts_tablet_manager.cc:1939] T 304e72b0c8c04c66924df2130b393543 P 5f9b6d5746fa40b5885d9db64ac86d3f: Deleting consensus metadata
I20260620 07:09:59.454212 24218 catalog_manager.cc:4841] TS 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): tablet 304e72b0c8c04c66924df2130b393543 (table test-table [id=853bc2048f7346ac92f49059469f65c2]) successfully deleted
W20260620 07:09:59.455394 24218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:36761 (error 108)
I20260620 07:09:59.455955 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24426
I20260620 07:09:59.456744 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:09:59.456144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:36761 (local address 127.0.0.1:60722)
0620 07:09:59.456581 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0620 07:09:59.456665 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323}
W20260620 07:09:59.456889 24218 catalog_manager.cc:4584] TS f831000e02fd4e2da033e2c6baf548ab (127.8.24.1:36761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 304e72b0c8c04c66924df2130b393543: Network error: Client connection negotiation failed: client connection to 127.8.24.1:36761: connect: Connection refused (error 111)
I20260620 07:09:59.461073 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24563
I20260620 07:09:59.465372 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24200
I20260620 07:10:19.470655 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--webserver_interface=127.8.24.62
--webserver_port=45449
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:19.578562 24728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:19.578797 24728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:19.578840 24728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:19.582777 24728 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 07:10:19.582828 24728 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 07:10:19.582842 24728 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 07:10:19.582867 24728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:19.582896 24728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:19.582926 24728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:19.582952 24728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:19.582983 24728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:19.588572 24728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:19.589582 24728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:19.590581 24728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:19.598613 24733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.598659 24734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.598613 24736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:19.599022 24728 server_base.cc:1034] running on GCE node
I20260620 07:10:19.599668 24728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:19.600860 24728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:19.602011 24728 hybrid_clock.cc:648] HybridClock initialized: now 1781939419601995 us; error 27 us; skew 500 ppm
Jun 20 07:10:19 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:19.604709 24728 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:10:19.605549 24728 webserver.cc:460] Webserver started at http://127.8.24.62:45449/ using document root <none> and password file <none>
I20260620 07:10:19.606050 24728 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:19.606106 24728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:19.609884 24728 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:19.611851 24743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:19.612991 24728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:19.613086 24728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ae76e7ce9b7f4b58b56f92b907238a5c"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "17dd728ecc187cddc37684722b64ad81"
server_key_iv: "abcde1b99f5070621e35d9e9a676fc5e"
server_key_version: "encryptionkey@0"
I20260620 07:10:19.613593 24728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:19.624173 24728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:19.626573 24728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:19.626715 24728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:19.632968 24728 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38805
I20260620 07:10:19.633226 24795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38805 every 8 connection(s)
I20260620 07:10:19.633970 24728 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:19.636272 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24728
I20260620 07:10:19.636655 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:36761
--local_ip_for_outbound_sockets=127.8.24.1
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=38165
--webserver_interface=127.8.24.1
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:10:19.638172 24796 sys_catalog.cc:263] Verifying existing consensus state
I20260620 07:10:19.639991 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap starting.
I20260620 07:10:19.656160 24796 log.cc:824] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:19.666625 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:10:19.666966 24796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: Bootstrap complete.
I20260620 07:10:19.670688 24796 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:10:19.671057 24796 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Initialized, Role: FOLLOWER
I20260620 07:10:19.671537 24796 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:10:19.671665 24796 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:19.671728 24796 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:19.671819 24796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 2 FOLLOWER]: Advancing to term 3
I20260620 07:10:19.673561 24796 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:10:19.673861 24796 leader_election.cc:302] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae76e7ce9b7f4b58b56f92b907238a5c; no voters:
I20260620 07:10:19.674132 24796 leader_election.cc:288] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 07:10:19.674229 24800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 3 FOLLOWER]: Leader election won for term 3
I20260620 07:10:19.674419 24800 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [term 3 LEADER]: Becoming Leader. State: Replica: ae76e7ce9b7f4b58b56f92b907238a5c, State: Running, Role: LEADER
I20260620 07:10:19.674687 24800 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } }
I20260620 07:10:19.675298 24796 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:19.675642 24800 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae76e7ce9b7f4b58b56f92b907238a5c. Latest consensus state: current_term: 3 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:10:19.676342 24800 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:19.676749 24801 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae76e7ce9b7f4b58b56f92b907238a5c" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38805 } } }
I20260620 07:10:19.676990 24801 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:19.679525 24816 catalog_manager.cc:1259] Loaded cluster ID: 9301fe9fb3cf4226847b097871078432
I20260620 07:10:19.679565 24816 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: loading cluster ID for follower catalog manager: success
I20260620 07:10:19.681273 24816 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae76e7ce9b7f4b58b56f92b907238a5c: acquiring CA information for follower catalog manager: success
F20260620 07:10:19.682142 24816 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2bd35dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f2bd35e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f2bd35dd4de google::LogMessage::Flush() at ??:0
@ 0x7f2bd35dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2bd62a118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2bddf42989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f2bddf1f023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f2bddf1f54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f2bddf18652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f2bddf179e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f2bddf5e541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2bd5820512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2bd6e646db start_thread at ??:0
@ 0x7f2bd20db61f clone at ??:0
W20260620 07:10:19.763743 24798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:19.763973 24798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:19.764017 24798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:19.767939 24798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:19.767990 24798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:19.768046 24798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:19.773097 24798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:36761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=38165
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:19.774080 24798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:19.775035 24798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:19.785058 24837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.785089 24836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.785064 24839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:19.785342 24798 server_base.cc:1034] running on GCE node
I20260620 07:10:19.785836 24798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:19.786603 24798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:19.787786 24798 hybrid_clock.cc:648] HybridClock initialized: now 1781939419787758 us; error 39 us; skew 500 ppm
Jun 20 07:10:19 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:19.790940 24798 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:10:19.791760 24798 webserver.cc:460] Webserver started at http://127.8.24.1:38165/ using document root <none> and password file <none>
I20260620 07:10:19.792301 24798 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:19.792361 24798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:19.796118 24798 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:19.798213 24846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:19.799486 24798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:19.799579 24798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "f831000e02fd4e2da033e2c6baf548ab"
format_stamp: "Formatted at 2026-06-20 07:09:37 on dist-test-slave-wjw3"
server_key: "d76744b8ba4ee9655fe261046cdb8a7b"
server_key_iv: "064f1160d239e124e99376bfdeea632a"
server_key_version: "encryptionkey@0"
I20260620 07:10:19.800107 24798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:19.813194 24798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:19.816111 24798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:19.816270 24798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:19.816799 24798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:19.817868 24853 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 07:10:19.822940 24798 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 07:10:19.822992 24798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260620 07:10:19.823138 24798 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 07:10:19.824793 24798 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 07:10:19.824836 24798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:19.825148 24853 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap starting.
I20260620 07:10:19.834038 24798 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:36761
I20260620 07:10:19.834290 24960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:36761 every 8 connection(s)
I20260620 07:10:19.834978 24798 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:19.843102 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24798
I20260620 07:10:19.843513 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:32825
--local_ip_for_outbound_sockets=127.8.24.2
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=34319
--webserver_interface=127.8.24.2
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:10:19.846478 24853 log.cc:824] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:19.855072 24853 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 07:10:19.855464 24853 tablet_bootstrap.cc:492] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Bootstrap complete.
I20260620 07:10:19.856739 24853 ts_tablet_manager.cc:1397] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.005s
I20260620 07:10:19.860003 24853 raft_consensus.cc:348] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:19.860519 24853 raft_consensus.cc:729] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f831000e02fd4e2da033e2c6baf548ab, State: Initialized, Role: FOLLOWER
I20260620 07:10:19.861042 24853 consensus_queue.cc:260] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:19.862116 24853 ts_tablet_manager.cc:1428] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260620 07:10:19.874047 24963 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:10:19.836788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:54605)
0620 07:10:19.837243 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:19.837294 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:19.838073 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:19.873167 (+ 35094us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
W20260620 07:10:19.874534 24961 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:38805 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:38805: Client connection negotiation failed: client connection to 127.8.24.62:38805: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 07:10:19.875211 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:19.874951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:46301)
0620 07:10:19.875080 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:19.875150 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 07:10:19.875811 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:19.875609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:55557)
0620 07:10:19.875707 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:19.875756 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 07:10:19.875960 24961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:10:19.950986 24964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:19.951215 24964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:19.951258 24964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:19.955165 24964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:19.955267 24964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:19.955332 24964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:19.960361 24964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:32825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=34319
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:19.961299 24964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:19.962242 24964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:19.971100 24972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.971146 24974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:19.971098 24971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:19.971306 24964 server_base.cc:1034] running on GCE node
I20260620 07:10:19.971722 24964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:19.972290 24964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:19.973438 24964 hybrid_clock.cc:648] HybridClock initialized: now 1781939419973424 us; error 32 us; skew 500 ppm
Jun 20 07:10:19 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:19.975939 24964 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:10:19.976724 24964 webserver.cc:460] Webserver started at http://127.8.24.2:34319/ using document root <none> and password file <none>
I20260620 07:10:19.977206 24964 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:19.977260 24964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:19.980873 24964 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:19.982697 24981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:19.983843 24964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:19.983935 24964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "c4d35f5102734f5793671129e650bdb6"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "963538d6c15878d03178273c60a79ace"
server_key_iv: "63b98cff6f3dcb99abbca34585ecd39d"
server_key_version: "encryptionkey@0"
I20260620 07:10:19.984439 24964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:20.002561 24964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:20.005297 24964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:20.005457 24964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:20.006014 24964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:20.006773 24964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:20.006817 24964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:20.006860 24964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:20.006887 24964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:20.015161 24964 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:32825
I20260620 07:10:20.015188 25094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:32825 every 8 connection(s)
I20260620 07:10:20.016085 24964 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:20.019682 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.018018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:53561)
0620 07:10:20.018441 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.018542 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
W20260620 07:10:20.020058 25095 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:38805 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:38805: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
I20260620 07:10:20.020169 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 24964
I20260620 07:10:20.020586 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.020397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:53825)
0620 07:10:20.020491 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.020533 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 07:10:20.020613 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:46497
--local_ip_for_outbound_sockets=127.8.24.3
--tserver_master_addrs=127.8.24.62:38805
--webserver_port=38119
--webserver_interface=127.8.24.3
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:10:20.021231 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.021048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:56357)
0620 07:10:20.021144 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.021184 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 07:10:20.021435 25095 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 07:10:20.123941 25098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:20.124173 25098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:20.124231 25098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:20.128127 25098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:20.128181 25098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:20.128255 25098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:20.133242 25098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:46497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=38119
--tserver_master_addrs=127.8.24.62:38805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:20.134181 25098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:20.135114 25098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:10:20.144045 25098 server_base.cc:1034] running on GCE node
W20260620 07:10:20.144017 25103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:20.144001 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:20.144037 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:20.144665 25098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:20.145232 25098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:20.146370 25098 hybrid_clock.cc:648] HybridClock initialized: now 1781939420146343 us; error 47 us; skew 500 ppm
Jun 20 07:10:20 dist-test-slave-wjw3 krb5kdc[23652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939420, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:20.148761 25098 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:10:20.149550 25098 webserver.cc:460] Webserver started at http://127.8.24.3:38119/ using document root <none> and password file <none>
I20260620 07:10:20.150040 25098 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:20.150100 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:20.153625 25098 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:20.155470 25113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:20.156577 25098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:20.156672 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "5f9b6d5746fa40b5885d9db64ac86d3f"
format_stamp: "Formatted at 2026-06-20 07:09:38 on dist-test-slave-wjw3"
server_key: "9bad1af79bebcc8b3c38cdfa070b5ea4"
server_key_iv: "7e0da545dfe21ec06e8a311bd9131e64"
server_key_version: "encryptionkey@0"
I20260620 07:10:20.157160 25098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:20.171252 25098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:20.173673 25098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:20.173823 25098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:20.174325 25098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:20.175061 25098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:20.175107 25098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:20.175163 25098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:20.175189 25098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:20.183166 25098 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:46497
I20260620 07:10:20.183182 25226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:46497 every 8 connection(s)
I20260620 07:10:20.184101 25098 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:20.186029 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25098
I20260620 07:10:20.187906 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.186031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:40869)
0620 07:10:20.186500 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.186624 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":119,"threads_started":1}
W20260620 07:10:20.188328 25227 heartbeater.cc:644] Failed to heartbeat to 127.8.24.62:38805 (0 consecutive failures): Network error: Failed to ping master at 127.8.24.62:38805: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
I20260620 07:10:20.188838 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.188642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:34401)
0620 07:10:20.188742 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.188787 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 07:10:20.189332 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.189154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:60193)
0620 07:10:20.189243 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.189285 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 07:10:20.189477 25227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 07:10:20.876838 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:20.876454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:34873)
0620 07:10:20.876683 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:20.876733 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260620 07:10:21.022233 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.021884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:33563)
0620 07:10:21.022090 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:21.022139 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260620 07:10:21.190311 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.189953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:51035)
0620 07:10:21.190158 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:21.190209 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":63,"threads_started":1}
I20260620 07:10:21.215799 25234 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:21.215994 25234 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:21.216981 25234 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
Jun 20 07:10:21 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.2@KRBTEST.COM
Jun 20 07:10:21 dist-test-slave-wjw3 krb5kdc[23652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.3@KRBTEST.COM
I20260620 07:10:21.232509 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.217322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:43305 (local address 127.8.24.3:46497)
0620 07:10:21.217722 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:21.217748 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:21.217795 (+ 47us) server_negotiation.cc:407] Connection header received
0620 07:10:21.218685 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:21.218694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:21.219152 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:21.219503 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:21.221251 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.222839 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.224697 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.225427 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.225603 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:21.228658 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:21.228663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:21.228673 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:21.228702 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:21.230244 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230649 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:21.230653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:21.230657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:21.230762 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:21.231014 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:21.231017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:21.231018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:21.231398 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:21.231557 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:21.232085 (+ 528us) server_negotiation.cc:299] Negotiation successful
0620 07:10:21.232327 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":116,"threads_started":1}
I20260620 07:10:21.232509 25236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.217426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:32825 (local address 127.8.24.1:57759)
0620 07:10:21.217729 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:21.217743 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:21.217827 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:21.219600 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:21.219607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:21.220251 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:21.221067 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:21.221089 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.222988 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.222992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:21.224592 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:21.224601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.225538 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.225541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:21.225813 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:21.226458 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:21.226480 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:21.228243 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:21.230103 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:21.230122 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:21.230353 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:21.230673 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:21.230678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:21.230867 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:21.231526 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:21.231532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:21.231784 (+ 252us) client_negotiation.cc:769] Sending connection context
0620 07:10:21.231944 (+ 160us) client_negotiation.cc:240] Negotiation successful
0620 07:10:21.232311 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":370,"spinlock_wait_cycles":40064,"thread_start_us":65,"threads_started":1}
I20260620 07:10:21.232545 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.217185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:46497 (local address 127.8.24.1:43305)
0620 07:10:21.217308 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:21.217322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:21.217434 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:21.219600 (+ 2166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:21.219607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:21.220251 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:21.221067 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:21.221088 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.222988 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.222992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:21.224591 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:21.224601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.225538 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.225541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:21.225813 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:21.226491 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:21.226503 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:21.228480 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:21.230349 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:21.230354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:21.230537 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:21.230849 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:21.230854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:21.230928 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:21.231530 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:21.231533 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:21.231780 (+ 247us) client_negotiation.cc:769] Sending connection context
0620 07:10:21.231944 (+ 164us) client_negotiation.cc:240] Negotiation successful
0620 07:10:21.232387 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":439,"spinlock_wait_cycles":8320}
I20260620 07:10:21.232515 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.217598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57759 (local address 127.8.24.2:32825)
0620 07:10:21.217888 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:21.217909 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:21.217940 (+ 31us) server_negotiation.cc:407] Connection header received
0620 07:10:21.218685 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:21.218694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:21.219152 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:21.219502 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:21.221267 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.222839 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.224697 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:21.225427 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:21.225596 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:21.228449 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:21.228454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:21.228464 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:21.228493 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:21.229998 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:21.230464 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:21.230469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:21.230473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:21.230579 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:21.231000 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:21.231004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:21.231005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:21.231400 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:21.231568 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:21.232090 (+ 522us) server_negotiation.cc:299] Negotiation successful
0620 07:10:21.232339 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":100,"threads_started":1}
W20260620 07:10:21.235019 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
W20260620 07:10:21.235239 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
I20260620 07:10:21.235292 24848 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f831000e02fd4e2da033e2c6baf548ab; no voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6
I20260620 07:10:21.235543 25234 raft_consensus.cc:2726] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:10:21.877869 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:21.877501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:51707)
0620 07:10:21.877721 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:21.877774 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260620 07:10:22.023214 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:22.022837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:47765)
0620 07:10:22.023075 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:22.023132 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
I20260620 07:10:22.191282 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:22.190914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:52783)
0620 07:10:22.191147 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:22.191199 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
I20260620 07:10:22.878926 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:22.878556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:58939)
0620 07:10:22.878783 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:22.878833 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260620 07:10:23.024251 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:23.023833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:48907)
0620 07:10:23.024079 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:23.024135 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260620 07:10:23.192243 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:23.191869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:41631)
0620 07:10:23.192103 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:23.192151 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260620 07:10:23.419765 25244 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:23.419891 25244 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:23.420286 25244 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
W20260620 07:10:23.421001 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
W20260620 07:10:23.421114 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
I20260620 07:10:23.421161 24848 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f831000e02fd4e2da033e2c6baf548ab; no voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6
I20260620 07:10:23.421305 25244 raft_consensus.cc:2726] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:10:23.880021 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:23.879649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:34915)
0620 07:10:23.879877 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:23.879928 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":79,"threads_started":1}
I20260620 07:10:24.025270 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:24.024868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:60999)
0620 07:10:24.025126 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:24.025181 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260620 07:10:24.193254 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:24.192845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:53027)
0620 07:10:24.193113 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:24.193168 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
I20260620 07:10:24.881465 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:24.880993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:58823)
0620 07:10:24.881301 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:24.881366 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":76,"threads_started":1}
I20260620 07:10:25.026520 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:25.026115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:38261)
0620 07:10:25.026372 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:25.026433 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
I20260620 07:10:25.139087 25250 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:25.139218 25250 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:25.139655 25250 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
W20260620 07:10:25.140405 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
W20260620 07:10:25.140543 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
I20260620 07:10:25.140606 24848 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f831000e02fd4e2da033e2c6baf548ab; no voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6
I20260620 07:10:25.140774 25250 raft_consensus.cc:2726] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:10:25.194361 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:25.193968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:45961)
0620 07:10:25.194219 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:25.194278 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260620 07:10:25.882508 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:25.882146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:54179)
0620 07:10:25.882366 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:25.882414 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
I20260620 07:10:26.027585 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:26.027174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:58923)
0620 07:10:26.027401 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:26.027491 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
I20260620 07:10:26.195314 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:26.194959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:34585)
0620 07:10:26.195172 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:26.195222 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":68,"threads_started":1}
I20260620 07:10:26.883626 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:26.883192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:36783)
0620 07:10:26.883458 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:26.883522 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260620 07:10:27.028677 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:27.028239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:41845)
0620 07:10:27.028530 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:27.028581 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":81,"threads_started":1}
I20260620 07:10:27.196347 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:27.195912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:33255)
0620 07:10:27.196210 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:27.196261 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":82,"threads_started":1}
I20260620 07:10:27.884666 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:27.884251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:50171)
0620 07:10:27.884498 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:27.884553 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260620 07:10:28.029663 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:28.029286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:46361)
0620 07:10:28.029527 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:28.029577 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260620 07:10:28.197366 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:28.197001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:38085)
0620 07:10:28.197225 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:28.197275 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260620 07:10:28.885723 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:28.885330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:60427)
0620 07:10:28.885563 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:28.885628 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260620 07:10:29.030802 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:29.030363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:44373)
0620 07:10:29.030652 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:29.030705 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":77,"threads_started":1}
I20260620 07:10:29.198387 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:29.197993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:43157)
0620 07:10:29.198249 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:29.198300 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":105,"threads_started":1}
I20260620 07:10:29.707516 25264 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:29.707671 25264 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:29.708055 25264 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
W20260620 07:10:29.708808 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
W20260620 07:10:29.708945 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
I20260620 07:10:29.708983 24848 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f831000e02fd4e2da033e2c6baf548ab; no voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6
I20260620 07:10:29.709117 25264 raft_consensus.cc:2726] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:10:29.886966 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:29.886501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:49275)
0620 07:10:29.886808 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:29.886870 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
I20260620 07:10:30.031958 25266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:30.031582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:54079)
0620 07:10:30.031811 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:30.031866 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260620 07:10:30.199357 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:30.199003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:55387)
0620 07:10:30.199218 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:30.199267 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260620 07:10:30.888077 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:30.887673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:49391)
0620 07:10:30.887924 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:30.887980 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":92,"threads_started":1}
I20260620 07:10:31.032903 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:31.032559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:55993)
0620 07:10:31.032770 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:31.032818 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260620 07:10:31.200398 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:31.200023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:45243)
0620 07:10:31.200258 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:31.200308 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260620 07:10:31.889158 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:31.888751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:52153)
0620 07:10:31.888991 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:31.889061 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260620 07:10:32.034008 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:32.033616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:60073)
0620 07:10:32.033857 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:32.033915 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260620 07:10:32.201376 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:32.200999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:57651)
0620 07:10:32.201231 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:32.201284 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260620 07:10:32.890220 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:32.889810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:40113)
0620 07:10:32.890067 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:32.890126 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260620 07:10:33.035033 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:33.034655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:35023)
0620 07:10:33.034886 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:33.034938 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260620 07:10:33.202399 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:33.202007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:52711)
0620 07:10:33.202258 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:33.202309 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260620 07:10:33.843974 25277 raft_consensus.cc:482] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:33.844095 25277 raft_consensus.cc:504] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f9b6d5746fa40b5885d9db64ac86d3f" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 46497 } } peers { permanent_uuid: "f831000e02fd4e2da033e2c6baf548ab" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 36761 } } peers { permanent_uuid: "c4d35f5102734f5793671129e650bdb6" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 32825 } }
I20260620 07:10:33.844485 25277 leader_election.cc:288] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497), c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825)
W20260620 07:10:33.845172 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f9b6d5746fa40b5885d9db64ac86d3f (127.8.24.3:46497): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
W20260620 07:10:33.845289 24848 leader_election.cc:341] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c4d35f5102734f5793671129e650bdb6 (127.8.24.2:32825): Not found: Tablet not found: 304e72b0c8c04c66924df2130b393543
I20260620 07:10:33.845340 24848 leader_election.cc:302] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f831000e02fd4e2da033e2c6baf548ab; no voters: 5f9b6d5746fa40b5885d9db64ac86d3f, c4d35f5102734f5793671129e650bdb6
I20260620 07:10:33.845461 25277 raft_consensus.cc:2726] T 304e72b0c8c04c66924df2130b393543 P f831000e02fd4e2da033e2c6baf548ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 07:10:33.891307 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:33.890915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:46259)
0620 07:10:33.891161 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:33.891214 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260620 07:10:34.036110 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:34.035706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:50759)
0620 07:10:34.035937 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:34.035997 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260620 07:10:34.203516 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:34.203067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:56983)
0620 07:10:34.203337 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:34.203389 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260620 07:10:34.892593 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:34.892187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.1:36255)
0620 07:10:34.892438 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:34.892494 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260620 07:10:35.037243 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:35.036868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.2:51391)
0620 07:10:35.037106 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:35.037153 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
I20260620 07:10:35.196128 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24798
I20260620 07:10:35.200982 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 24964
I20260620 07:10:35.204574 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:35.204221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38805 (local address 127.8.24.3:42655)
0620 07:10:35.204438 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:35.204487 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.62:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260620 07:10:35.205387 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25098
2026-06-20T07:10:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58804 ms)
[----------] 30 tests from SecurityITest (234862 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:35 dist-test-slave-wjw3 krb5kdc[25286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:35 dist-test-slave-wjw3 krb5kdc[25286](info): set up 2 sockets
Jun 20 07:10:35 dist-test-slave-wjw3 krb5kdc[25286](info): commencing operation
krb5kdc: starting...
W20260620 07:10:36.163055 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:10:36.270210 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45363
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:36.374133 25299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:36.374369 25299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:36.374418 25299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:36.378386 25299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:36.378439 25299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:36.378456 25299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:36.378474 25299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:36.378490 25299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:36.383482 25299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:36.384403 25299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:36.385368 25299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:36.393354 25304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:36.393362 25305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.393426 25299 server_base.cc:1034] running on GCE node
W20260620 07:10:36.393374 25307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.394102 25299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:36.395120 25299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:36.396265 25299 hybrid_clock.cc:648] HybridClock initialized: now 1781939436396245 us; error 32 us; skew 500 ppm
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:36.398716 25299 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:10:36.399521 25299 webserver.cc:460] Webserver started at http://127.8.24.62:45277/ using document root <none> and password file <none>
I20260620 07:10:36.400004 25299 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:36.400063 25299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:36.400224 25299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:36.402096 25299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "990839c795f5409aaad5e9cee150723d"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "0d5965fce04b33dfbc9302f6720578ab"
server_key_iv: "a4c0d90cbc16cfb43069ce092274e3bd"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.402578 25299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "990839c795f5409aaad5e9cee150723d"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "0d5965fce04b33dfbc9302f6720578ab"
server_key_iv: "a4c0d90cbc16cfb43069ce092274e3bd"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.405916 25299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:10:36.408051 25314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.409154 25299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:36.409245 25299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "990839c795f5409aaad5e9cee150723d"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "0d5965fce04b33dfbc9302f6720578ab"
server_key_iv: "a4c0d90cbc16cfb43069ce092274e3bd"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.409320 25299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:36.419301 25299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:36.421748 25299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:36.421883 25299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:36.428021 25299 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45363
I20260620 07:10:36.428037 25366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45363 every 8 connection(s)
I20260620 07:10:36.428855 25299 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:36.431337 25367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:36.435364 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25299
I20260620 07:10:36.435484 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:36.435750 8288 external_mini_cluster.cc:1428] Setting key 27734fd6ca6119f596b928dc582f5281
I20260620 07:10:36.436722 25367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: Bootstrap starting.
I20260620 07:10:36.438879 25367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:36.439720 25367 log.cc:824] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:36.441669 25367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: No bootstrap required, opened a new log
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:36.445727 25367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } }
I20260620 07:10:36.446007 25367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:36.446065 25367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990839c795f5409aaad5e9cee150723d, State: Initialized, Role: FOLLOWER
I20260620 07:10:36.446539 25367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } }
I20260620 07:10:36.446691 25367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:36.446761 25367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:36.446856 25367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:36.447881 25367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } }
I20260620 07:10:36.448222 25367 leader_election.cc:302] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 990839c795f5409aaad5e9cee150723d; no voters:
I20260620 07:10:36.448537 25367 leader_election.cc:288] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:36.448858 25373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:36.449616 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:36.437016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.8.24.62:45363)
0620 07:10:36.437387 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:36.437402 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:36.437436 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:10:36.438042 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:36.438068 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:36.438382 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:36.438648 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:36.439532 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.440676 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.441758 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.442229 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.442339 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:36.444870 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:36.444877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:36.444893 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:36.444924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:36.446674 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.447091 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.447097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.447103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.447230 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.447457 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.447460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.447462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.447960 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:36.448141 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:36.448422 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 07:10:36.448745 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260620 07:10:36.449685 25367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:36.449035 25373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [term 1 LEADER]: Becoming Leader. State: Replica: 990839c795f5409aaad5e9cee150723d, State: Running, Role: LEADER
I20260620 07:10:36.450171 25373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } }
I20260620 07:10:36.451700 25372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 990839c795f5409aaad5e9cee150723d. Latest consensus state: current_term: 1 leader_uuid: "990839c795f5409aaad5e9cee150723d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } } }
I20260620 07:10:36.451918 25374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "990839c795f5409aaad5e9cee150723d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990839c795f5409aaad5e9cee150723d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45363 } } }
I20260620 07:10:36.451968 25372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:36.452008 25374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d [sys.catalog]: This master's current role is: LEADER
W20260620 07:10:36.454416 25386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:10:36.454476 25386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:10:36.454540 25387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:36.455212 25387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:36.460351 25387 catalog_manager.cc:1347] Generated new cluster ID: b00a9b68259b42149a319ff7fb7092d7
I20260620 07:10:36.460417 25387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:36.470754 25387 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:36.471688 25387 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:36.476013 25387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: Generated new TSK 0
I20260620 07:10:36.476629 25387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:10:36.501789 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:36.606757 25393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:36.606981 25393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:36.607021 25393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:36.610936 25393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:36.611001 25393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:36.611066 25393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:36.616369 25393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:36.617408 25393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:36.618407 25393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:36.627195 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:36.627205 25401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.627272 25393 server_base.cc:1034] running on GCE node
W20260620 07:10:36.627210 25399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.627808 25393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:36.628345 25393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:36.629530 25393 hybrid_clock.cc:648] HybridClock initialized: now 1781939436629510 us; error 28 us; skew 500 ppm
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:36.632197 25393 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:10:36.632992 25393 webserver.cc:460] Webserver started at http://127.8.24.1:33099/ using document root <none> and password file <none>
I20260620 07:10:36.633518 25393 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:36.633584 25393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:36.633767 25393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:36.635722 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "dd9f36bb6d1f4909b5637993ccf39c04"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "f6c51cbe1ca94b6cb0a5c224dbfba29f"
server_key_iv: "0cf7b28e38f453b19fe15abe0ad612a8"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.636216 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "dd9f36bb6d1f4909b5637993ccf39c04"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "f6c51cbe1ca94b6cb0a5c224dbfba29f"
server_key_iv: "0cf7b28e38f453b19fe15abe0ad612a8"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.639665 25393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:10:36.641850 25408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.642967 25393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:36.643076 25393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "dd9f36bb6d1f4909b5637993ccf39c04"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "f6c51cbe1ca94b6cb0a5c224dbfba29f"
server_key_iv: "0cf7b28e38f453b19fe15abe0ad612a8"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.643163 25393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:36.659044 25393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:36.661572 25393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:36.661728 25393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:36.662256 25393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:36.662998 25393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:36.663048 25393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.663105 25393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:36.663130 25393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.670814 25393 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:34935
I20260620 07:10:36.670869 25521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:34935 every 8 connection(s)
I20260620 07:10:36.671644 25393 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:36.677567 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25393
I20260620 07:10:36.677685 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:36.677937 8288 external_mini_cluster.cc:1428] Setting key dcef3694368361469a8fe80ef1d188b5
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:36.684152 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:36.673238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:42955 (local address 127.8.24.62:45363)
0620 07:10:36.673369 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:36.673374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:36.674148 (+ 774us) server_negotiation.cc:407] Connection header received
0620 07:10:36.675299 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:36.675301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:36.675346 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:36.675412 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:36.676471 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.676986 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.678189 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.678471 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.678575 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:36.681116 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:36.681118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:36.681120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:36.681142 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:36.682375 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.682813 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.682817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.682818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.682888 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.683179 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.683183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.683184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.683345 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:36.683425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:36.683884 (+ 459us) server_negotiation.cc:299] Negotiation successful
0620 07:10:36.683997 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:10:36.684863 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:36.673582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45363 (local address 127.8.24.1:42955)
0620 07:10:36.674019 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:36.674059 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:36.675126 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:36.675585 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:36.675592 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:36.675913 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:36.676363 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:36.676376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.677069 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.677071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:36.677986 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:36.677996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.678605 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.678609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:36.678791 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:36.679357 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:36.679374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:36.680929 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:36.682479 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:36.682485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:36.682498 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:36.682725 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:36.682965 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:36.682968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:36.682970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:36.683099 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:36.683455 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:36.683461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:36.683674 (+ 213us) client_negotiation.cc:769] Sending connection context
0620 07:10:36.683827 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 07:10:36.684122 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
I20260620 07:10:36.685930 25522 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45363
I20260620 07:10:36.686136 25522 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:10:36.686578 25522 heartbeater.cc:511] Master 127.8.24.62:45363 requested a full tablet report, sending...
I20260620 07:10:36.687956 25331 ts_manager.cc:194] Registered new tserver with Master: dd9f36bb6d1f4909b5637993ccf39c04 (127.8.24.1:34935)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:10:36.689100 25331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:42955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:10:36.701939 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:36.807104 25527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:36.807325 25527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:36.807370 25527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:36.811244 25527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:36.811296 25527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:36.811367 25527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:36.816592 25527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:36.817564 25527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:36.818538 25527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:36.827570 25533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.827639 25527 server_base.cc:1034] running on GCE node
W20260620 07:10:36.827631 25532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:36.827571 25535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:36.828150 25527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:36.828707 25527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:36.829852 25527 hybrid_clock.cc:648] HybridClock initialized: now 1781939436829835 us; error 29 us; skew 500 ppm
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:36.832329 25527 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:10:36.833145 25527 webserver.cc:460] Webserver started at http://127.8.24.2:42223/ using document root <none> and password file <none>
I20260620 07:10:36.833657 25527 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:36.833712 25527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:36.833868 25527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:36.835800 25527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "cbdc4e921f504174a1c034e1c67eca1a"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "1bd02e83115cb4ac205cb71cc505dc49"
server_key_iv: "f8b6ac2162568490689501e2061007a3"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.836282 25527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "cbdc4e921f504174a1c034e1c67eca1a"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "1bd02e83115cb4ac205cb71cc505dc49"
server_key_iv: "f8b6ac2162568490689501e2061007a3"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.839650 25527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:36.841769 25542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.842878 25527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:36.842973 25527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "cbdc4e921f504174a1c034e1c67eca1a"
format_stamp: "Formatted at 2026-06-20 07:10:36 on dist-test-slave-wjw3"
server_key: "1bd02e83115cb4ac205cb71cc505dc49"
server_key_iv: "f8b6ac2162568490689501e2061007a3"
server_key_version: "encryptionkey@0"
I20260620 07:10:36.843073 25527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:36.860471 25527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:36.863085 25527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:36.863234 25527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:36.863775 25527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:36.864516 25527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:36.864559 25527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.864600 25527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:36.864629 25527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:36.872627 25527 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38789
I20260620 07:10:36.872654 25655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38789 every 8 connection(s)
I20260620 07:10:36.873410 25527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:36.877451 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25527
I20260620 07:10:36.877583 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:36.877830 8288 external_mini_cluster.cc:1428] Setting key 31fa04a93b769e860a769d36ef2ff663
Jun 20 07:10:36 dist-test-slave-wjw3 krb5kdc[25286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:10:36.887477 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:36.874979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:59853 (local address 127.8.24.62:45363)
0620 07:10:36.875093 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:36.875096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:36.875851 (+ 755us) server_negotiation.cc:407] Connection header received
0620 07:10:36.877084 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:36.877087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:36.877150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:36.877246 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:36.878500 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.879013 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.880148 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.880354 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.880437 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:36.882892 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:36.882895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:36.882897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:36.882958 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:36.884828 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.885512 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.885517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.885519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.885610 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:36.885990 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:36.885994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:36.885996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:36.886244 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:36.886348 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:36.887159 (+ 811us) server_negotiation.cc:299] Negotiation successful
0620 07:10:36.887294 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:36.888633 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:36.875283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45363 (local address 127.8.24.2:59853)
0620 07:10:36.875708 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:36.875748 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:36.876878 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:36.877434 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:36.877441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:36.877810 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:36.878349 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:36.878369 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.879122 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.879125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:36.880031 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:36.880041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:36.880486 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:36.880489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:36.880637 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:36.881140 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:36.881155 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:36.882721 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:36.884952 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:36.884959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:36.884973 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:36.885253 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:36.885727 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:36.885730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:36.885732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:36.885900 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:36.886361 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:36.886368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:36.886831 (+ 463us) client_negotiation.cc:769] Sending connection context
0620 07:10:36.887047 (+ 216us) client_negotiation.cc:240] Negotiation successful
0620 07:10:36.887377 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
I20260620 07:10:36.889779 25656 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45363
I20260620 07:10:36.889989 25656 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:36.890465 25656 heartbeater.cc:511] Master 127.8.24.62:45363 requested a full tablet report, sending...
I20260620 07:10:36.891537 25331 ts_manager.cc:194] Registered new tserver with Master: cbdc4e921f504174a1c034e1c67eca1a (127.8.24.2:38789)
I20260620 07:10:36.892107 25331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:59853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:10:36.906908 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:37.013329 25661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:37.013572 25661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:37.013615 25661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:37.017519 25661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:37.017563 25661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:37.017628 25661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:37.022504 25661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:37.023399 25661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:37.024474 25661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:10:37.033257 25661 server_base.cc:1034] running on GCE node
W20260620 07:10:37.033214 25666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:37.033208 25667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:37.033196 25669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:37.033764 25661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:37.034296 25661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:37.035472 25661 hybrid_clock.cc:648] HybridClock initialized: now 1781939437035453 us; error 32 us; skew 500 ppm
Jun 20 07:10:37 dist-test-slave-wjw3 krb5kdc[25286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939437, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:37.038030 25661 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:10:37.038885 25661 webserver.cc:460] Webserver started at http://127.8.24.3:42777/ using document root <none> and password file <none>
I20260620 07:10:37.039403 25661 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:37.039499 25661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:37.039700 25661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:37.041651 25661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "33882b41f05a4ee68443e60d17873c28"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "d720cbde238a2dae5e0e458e092ebe0e"
server_key_iv: "d98a2c64f00569f5290a0da48398456c"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.042179 25661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "33882b41f05a4ee68443e60d17873c28"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "d720cbde238a2dae5e0e458e092ebe0e"
server_key_iv: "d98a2c64f00569f5290a0da48398456c"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.045732 25661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:10:37.048097 25676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.049203 25661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:37.049301 25661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "33882b41f05a4ee68443e60d17873c28"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "d720cbde238a2dae5e0e458e092ebe0e"
server_key_iv: "d98a2c64f00569f5290a0da48398456c"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.049384 25661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:37.060171 25661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:37.062783 25661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:37.062950 25661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:37.063549 25661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:37.064322 25661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:37.064368 25661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.064417 25661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:37.064445 25661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.072448 25661 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:42313
I20260620 07:10:37.072486 25789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:42313 every 8 connection(s)
I20260620 07:10:37.073315 25661 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
Jun 20 07:10:37 dist-test-slave-wjw3 krb5kdc[25286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939437, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:37.082836 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25661
I20260620 07:10:37.082968 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:37.083256 8288 external_mini_cluster.cc:1428] Setting key fd0ae1f409a0078474246fa423049424
I20260620 07:10:37.085268 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.074958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:42659 (local address 127.8.24.62:45363)
0620 07:10:37.075092 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.075096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.075911 (+ 815us) server_negotiation.cc:407] Connection header received
0620 07:10:37.076764 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.076768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.076823 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.076897 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:37.078010 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.078540 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.079668 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.079884 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.079969 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.082066 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:37.082069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:37.082071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:37.082095 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:37.083413 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:37.083863 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:37.083867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:37.083869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:37.083943 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:37.084218 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:37.084221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:37.084223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:37.084417 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:37.084508 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.085012 (+ 504us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.085119 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:10:37.086107 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.075313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45363 (local address 127.8.24.3:42659)
0620 07:10:37.075776 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.075817 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.076574 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.077075 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.077082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.077411 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:37.077875 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.077888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.078652 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.078655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.079563 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.079572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.080017 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.080021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.080177 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.080642 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:37.080658 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:37.081911 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:37.083530 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:37.083534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:37.083545 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:37.083780 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:37.084022 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:37.084025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:37.084026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:37.084151 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:37.084512 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:37.084517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:37.084792 (+ 275us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.085007 (+ 215us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.085328 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":159,"threads_started":1}
I20260620 07:10:37.087062 25790 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45363
I20260620 07:10:37.087257 25790 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:37.087692 25790 heartbeater.cc:511] Master 127.8.24.62:45363 requested a full tablet report, sending...
I20260620 07:10:37.088564 25331 ts_manager.cc:194] Registered new tserver with Master: 33882b41f05a4ee68443e60d17873c28 (127.8.24.3:42313)
I20260620 07:10:37.089069 25331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:42659
I20260620 07:10:37.096801 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:37.104421 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.098463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40338 (local address 127.8.24.62:45363)
0620 07:10:37.098591 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.098595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.098711 (+ 116us) server_negotiation.cc:407] Connection header received
0620 07:10:37.098796 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.098798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.098833 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.098887 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:37.099428 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.099989 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.100736 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.100927 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.100987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.102081 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:37.102084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:37.102086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:37.102107 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:37.103129 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:37.103496 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:37.103499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:37.103500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:37.103565 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:37.103750 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:37.103751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:37.103753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:37.103933 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:37.104023 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.104197 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.104280 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 07:10:37.106813 25331 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:37.109687 25331 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:37.123039 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.117969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34935 (local address 127.0.0.1:60558)
0620 07:10:37.118590 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.118603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.118683 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.119296 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.119299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.119316 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:37.119397 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.119403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.120734 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.120738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.121639 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.121648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.122531 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.122635 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.122671 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.122769 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.122896 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":15,"thread_start_us":87,"threads_started":1}
I20260620 07:10:37.123044 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.118163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.8.24.1:34935)
0620 07:10:37.118934 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.118945 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.118968 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:10:37.119041 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.119046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.119185 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.119307 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:37.119548 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.120593 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.121751 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.122424 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122515 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.122649 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.122808 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.122896 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":119,"threads_started":1}
I20260620 07:10:37.123143 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.118461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.8.24.2:38789)
0620 07:10:37.118828 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.118840 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.118908 (+ 68us) server_negotiation.cc:407] Connection header received
0620 07:10:37.119044 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.119049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.119242 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.119378 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:37.119693 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.120490 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.121711 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.122347 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122441 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.122575 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.122894 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.122984 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":148,"threads_started":1}
I20260620 07:10:37.123389 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.118092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38789 (local address 127.0.0.1:60978)
0620 07:10:37.118838 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.118849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.118931 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.119499 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.119502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.119520 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:37.119596 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.119601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.120620 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.120623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.121602 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.121611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.122635 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.122722 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.122745 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.122810 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.123208 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":324,"thread_start_us":75,"threads_started":1}
I20260620 07:10:37.125763 25456 tablet_service.cc:1467] Processing CreateTablet for tablet 87602bd1a558495fb63ea0943cba0573 (DEFAULT_TABLE table=test-table [id=864dae50fcc9499b86ed068dce144902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:37.125764 25590 tablet_service.cc:1467] Processing CreateTablet for tablet 87602bd1a558495fb63ea0943cba0573 (DEFAULT_TABLE table=test-table [id=864dae50fcc9499b86ed068dce144902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:37.126619 25590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87602bd1a558495fb63ea0943cba0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:37.126618 25456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87602bd1a558495fb63ea0943cba0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:37.127862 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.117441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.8.24.3:42313)
0620 07:10:37.117748 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.117759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.117783 (+ 24us) server_negotiation.cc:407] Connection header received
0620 07:10:37.117865 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.117869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.118068 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.118222 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:37.118724 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.119820 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122139 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.122897 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.122981 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.123653 (+ 672us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.127628 (+ 3975us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.127721 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
I20260620 07:10:37.128109 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.117297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:42313 (local address 127.0.0.1:47832)
0620 07:10:37.117591 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.117632 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.117746 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.118418 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.118424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.118449 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:37.118595 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.118603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.119942 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.119945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.122024 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.122033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.123715 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.123718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.123805 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.123826 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.123889 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.123952 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260620 07:10:37.130831 25724 tablet_service.cc:1467] Processing CreateTablet for tablet 87602bd1a558495fb63ea0943cba0573 (DEFAULT_TABLE table=test-table [id=864dae50fcc9499b86ed068dce144902]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:37.131817 25724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87602bd1a558495fb63ea0943cba0573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:37.132396 25807 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Bootstrap starting.
I20260620 07:10:37.133618 25808 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Bootstrap starting.
I20260620 07:10:37.134893 25807 tablet_bootstrap.cc:654] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:37.135895 25807 log.cc:824] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:37.136035 25808 tablet_bootstrap.cc:654] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:37.136801 25808 log.cc:824] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:37.137588 25807 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: No bootstrap required, opened a new log
I20260620 07:10:37.137756 25807 ts_tablet_manager.cc:1397] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260620 07:10:37.138307 25811 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Bootstrap starting.
I20260620 07:10:37.138644 25808 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: No bootstrap required, opened a new log
I20260620 07:10:37.138808 25808 ts_tablet_manager.cc:1397] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260620 07:10:37.140277 25811 tablet_bootstrap.cc:654] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:37.141160 25811 log.cc:824] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:37.142292 25807 raft_consensus.cc:348] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.142292 25808 raft_consensus.cc:348] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.142551 25808 raft_consensus.cc:374] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:37.142551 25807 raft_consensus.cc:374] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:37.142608 25807 raft_consensus.cc:729] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9f36bb6d1f4909b5637993ccf39c04, State: Initialized, Role: FOLLOWER
I20260620 07:10:37.142614 25808 raft_consensus.cc:729] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbdc4e921f504174a1c034e1c67eca1a, State: Initialized, Role: FOLLOWER
I20260620 07:10:37.142621 25811 tablet_bootstrap.cc:492] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: No bootstrap required, opened a new log
I20260620 07:10:37.142771 25811 ts_tablet_manager.cc:1397] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:10:37.143038 25807 consensus_queue.cc:260] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.143038 25808 consensus_queue.cc:260] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.143811 25522 heartbeater.cc:503] Master 127.8.24.62:45363 was elected leader, sending a full tablet report...
I20260620 07:10:37.143971 25807 ts_tablet_manager.cc:1428] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 07:10:37.144003 25808 ts_tablet_manager.cc:1428] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260620 07:10:37.144873 25656 heartbeater.cc:503] Master 127.8.24.62:45363 was elected leader, sending a full tablet report...
I20260620 07:10:37.146267 25811 raft_consensus.cc:348] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.146523 25811 raft_consensus.cc:374] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:37.146595 25811 raft_consensus.cc:729] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33882b41f05a4ee68443e60d17873c28, State: Initialized, Role: FOLLOWER
I20260620 07:10:37.147096 25811 consensus_queue.cc:260] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.147696 25790 heartbeater.cc:503] Master 127.8.24.62:45363 was elected leader, sending a full tablet report...
I20260620 07:10:37.147991 25811 ts_tablet_manager.cc:1428] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260620 07:10:37.172683 25523 tablet.cc:2369] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:10:37.324203 25791 tablet.cc:2369] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:10:37.377851 25657 tablet.cc:2369] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:37.417143 25813 raft_consensus.cc:482] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:37.417337 25813 raft_consensus.cc:504] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.418447 25813 leader_election.cc:288] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33882b41f05a4ee68443e60d17873c28 (127.8.24.3:42313), cbdc4e921f504174a1c034e1c67eca1a (127.8.24.2:38789)
I20260620 07:10:37.422096 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.418790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:51141 (local address 127.8.24.2:38789)
0620 07:10:37.418917 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.418922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.419154 (+ 232us) server_negotiation.cc:407] Connection header received
0620 07:10:37.419283 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.419285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.419328 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.419398 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:37.419681 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.420180 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421223 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.421602 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421689 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.421751 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.421919 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.421988 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:10:37.422111 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.418704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38789 (local address 127.8.24.1:51141)
0620 07:10:37.419051 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.419064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.419176 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.419417 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.419420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.419458 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:37.419582 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.419588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.420285 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.420289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.421104 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.421113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.421713 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.421807 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.421825 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.421887 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.421956 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260620 07:10:37.422158 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.418704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:42313 (local address 127.8.24.1:47655)
0620 07:10:37.419179 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.419190 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.419291 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.419534 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.419537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.419552 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:37.419688 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.419694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.420373 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.420377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.421133 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.421141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.421752 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.421857 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.421874 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.421934 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.422000 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":58,"thread_start_us":81,"threads_started":1}
I20260620 07:10:37.422137 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.418866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:47655 (local address 127.8.24.3:42313)
0620 07:10:37.418971 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.418975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.419275 (+ 300us) server_negotiation.cc:407] Connection header received
0620 07:10:37.419387 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.419390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.419459 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.419526 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:37.419794 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.420275 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421246 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.421645 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.421733 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.421793 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.421961 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.422028 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:37.422762 25610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87602bd1a558495fb63ea0943cba0573" candidate_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdc4e921f504174a1c034e1c67eca1a" is_pre_election: true
I20260620 07:10:37.423038 25610 raft_consensus.cc:2454] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9f36bb6d1f4909b5637993ccf39c04 in term 0.
I20260620 07:10:37.423089 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87602bd1a558495fb63ea0943cba0573" candidate_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33882b41f05a4ee68443e60d17873c28" is_pre_election: true
I20260620 07:10:37.423377 25744 raft_consensus.cc:2454] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9f36bb6d1f4909b5637993ccf39c04 in term 0.
I20260620 07:10:37.423529 25412 leader_election.cc:302] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbdc4e921f504174a1c034e1c67eca1a, dd9f36bb6d1f4909b5637993ccf39c04; no voters:
I20260620 07:10:37.423821 25813 raft_consensus.cc:2781] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:37.423910 25813 raft_consensus.cc:482] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:37.423961 25813 raft_consensus.cc:3037] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:37.425071 25813 raft_consensus.cc:504] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.425383 25813 leader_election.cc:288] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [CANDIDATE]: Term 1 election: Requested vote from peers 33882b41f05a4ee68443e60d17873c28 (127.8.24.3:42313), cbdc4e921f504174a1c034e1c67eca1a (127.8.24.2:38789)
I20260620 07:10:37.425706 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87602bd1a558495fb63ea0943cba0573" candidate_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33882b41f05a4ee68443e60d17873c28"
I20260620 07:10:37.425750 25610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87602bd1a558495fb63ea0943cba0573" candidate_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdc4e921f504174a1c034e1c67eca1a"
I20260620 07:10:37.425806 25744 raft_consensus.cc:3037] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:37.425868 25610 raft_consensus.cc:3037] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:37.426790 25744 raft_consensus.cc:2454] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9f36bb6d1f4909b5637993ccf39c04 in term 1.
I20260620 07:10:37.426813 25610 raft_consensus.cc:2454] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9f36bb6d1f4909b5637993ccf39c04 in term 1.
I20260620 07:10:37.427075 25411 leader_election.cc:302] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33882b41f05a4ee68443e60d17873c28, dd9f36bb6d1f4909b5637993ccf39c04; no voters:
I20260620 07:10:37.427219 25813 raft_consensus.cc:2781] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:37.427455 25813 raft_consensus.cc:686] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [term 1 LEADER]: Becoming Leader. State: Replica: dd9f36bb6d1f4909b5637993ccf39c04, State: Running, Role: LEADER
I20260620 07:10:37.427762 25813 consensus_queue.cc:237] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } }
I20260620 07:10:37.430775 25331 catalog_manager.cc:5495] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd9f36bb6d1f4909b5637993ccf39c04 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd9f36bb6d1f4909b5637993ccf39c04" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 34935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 } health_report { overall_health: UNKNOWN } } }
I20260620 07:10:37.482093 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.479060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.8.24.1:34935)
0620 07:10:37.479171 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.479175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.479205 (+ 30us) server_negotiation.cc:407] Connection header received
0620 07:10:37.479276 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.479278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.479332 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.479410 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:37.479627 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.480131 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.480905 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.481130 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.481204 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.481416 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:37.481816 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:37.481902 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.481928 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.481989 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:10:37.487738 25744 raft_consensus.cc:1261] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 1 FOLLOWER]: Refusing update from remote peer dd9f36bb6d1f4909b5637993ccf39c04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:37.487738 25610 raft_consensus.cc:1261] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 1 FOLLOWER]: Refusing update from remote peer dd9f36bb6d1f4909b5637993ccf39c04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:37.488471 25818 consensus_queue.cc:1035] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbdc4e921f504174a1c034e1c67eca1a" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:37.488679 25813 consensus_queue.cc:1035] T 87602bd1a558495fb63ea0943cba0573 P dd9f36bb6d1f4909b5637993ccf39c04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33882b41f05a4ee68443e60d17873c28" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:37.495376 25821 mvcc.cc:205] Tried to move back new op lower bound from 7298823935942287360 to 7298823935706591232. Current Snapshot: MvccSnapshot[applied={T|T < 7298823935942287360}]
I20260620 07:10:37.503876 25330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:37.504032 25330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:37.506170 25330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 990839c795f5409aaad5e9cee150723d: Sending DeleteTablet for 3 replicas of tablet 87602bd1a558495fb63ea0943cba0573
I20260620 07:10:37.506966 25590 tablet_service.cc:1514] Processing DeleteTablet for tablet 87602bd1a558495fb63ea0943cba0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:37 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:60978
I20260620 07:10:37.506983 25456 tablet_service.cc:1514] Processing DeleteTablet for tablet 87602bd1a558495fb63ea0943cba0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:37 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:60558
I20260620 07:10:37.507148 25724 tablet_service.cc:1514] Processing DeleteTablet for tablet 87602bd1a558495fb63ea0943cba0573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:37 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:47832
I20260620 07:10:37.507503 25832 tablet_replica.cc:330] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: stopping tablet replica
I20260620 07:10:37.507629 25833 tablet_replica.cc:330] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: stopping tablet replica
I20260620 07:10:37.507894 25832 raft_consensus.cc:2229] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:37.507987 25833 raft_consensus.cc:2229] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:37.508137 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25393
I20260620 07:10:37.508833 25832 raft_consensus.cc:2258] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:37.508833 25833 raft_consensus.cc:2258] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:37.510088 25832 ts_tablet_manager.cc:1905] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:37.510372 25833 ts_tablet_manager.cc:1905] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:37.512319 25832 ts_tablet_manager.cc:1918] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:37.512399 25832 log.cc:1196] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/87602bd1a558495fb63ea0943cba0573
I20260620 07:10:37.512609 25833 ts_tablet_manager.cc:1918] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:37.512686 25833 log.cc:1196] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/87602bd1a558495fb63ea0943cba0573
I20260620 07:10:37.512691 25832 ts_tablet_manager.cc:1939] T 87602bd1a558495fb63ea0943cba0573 P cbdc4e921f504174a1c034e1c67eca1a: Deleting consensus metadata
I20260620 07:10:37.512996 25833 ts_tablet_manager.cc:1939] T 87602bd1a558495fb63ea0943cba0573 P 33882b41f05a4ee68443e60d17873c28: Deleting consensus metadata
I20260620 07:10:37.513724 25318 catalog_manager.cc:4841] TS cbdc4e921f504174a1c034e1c67eca1a (127.8.24.2:38789): tablet 87602bd1a558495fb63ea0943cba0573 (table test-table [id=864dae50fcc9499b86ed068dce144902]) successfully deleted
I20260620 07:10:37.513926 25317 catalog_manager.cc:4841] TS 33882b41f05a4ee68443e60d17873c28 (127.8.24.3:42313): tablet 87602bd1a558495fb63ea0943cba0573 (table test-table [id=864dae50fcc9499b86ed068dce144902]) successfully deleted
W20260620 07:10:37.517352 25317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:34935 (error 108)
I20260620 07:10:37.517571 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25527
I20260620 07:10:37.518636 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.518395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:34935 (local address 127.0.0.1:60568)
0620 07:10:37.518511 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.518570 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 07:10:37.518822 25317 catalog_manager.cc:4584] TS dd9f36bb6d1f4909b5637993ccf39c04 (127.8.24.1:34935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87602bd1a558495fb63ea0943cba0573: Network error: Client connection negotiation failed: client connection to 127.8.24.1:34935: connect: Connection refused (error 111)
I20260620 07:10:37.522320 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25661
I20260620 07:10:37.526569 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25299
2026-06-20T07:10:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2320 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T07:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:37Z Disabled control of system clock
I20260620 07:10:37.553076 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38361
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:37.656664 25838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:37.656913 25838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:37.656955 25838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:37.660966 25838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:37.661018 25838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:37.661033 25838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:37.661063 25838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:37.661088 25838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:37.665709 25838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38361
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:37.666731 25838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:37.668085 25838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:37.675910 25844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:37.676007 25838 server_base.cc:1034] running on GCE node
W20260620 07:10:37.675859 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:37.675887 25843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:37.676551 25838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:37.677538 25838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:37.678686 25838 hybrid_clock.cc:648] HybridClock initialized: now 1781939437678667 us; error 31 us; skew 500 ppm
I20260620 07:10:37.680346 25838 webserver.cc:460] Webserver started at http://127.8.24.62:39953/ using document root <none> and password file <none>
I20260620 07:10:37.680840 25838 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:37.680900 25838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:37.681077 25838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:37.682924 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "5313807ecc7b41aaa5662d9c11f030a4"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "686619c337c17323dc3adb4aa6377fdc"
server_key_iv: "14654dde74e0e6f5715bc25581275940"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.683427 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5313807ecc7b41aaa5662d9c11f030a4"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "686619c337c17323dc3adb4aa6377fdc"
server_key_iv: "14654dde74e0e6f5715bc25581275940"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.686666 25838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:10:37.688781 25852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.689841 25838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:37.689941 25838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "5313807ecc7b41aaa5662d9c11f030a4"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "686619c337c17323dc3adb4aa6377fdc"
server_key_iv: "14654dde74e0e6f5715bc25581275940"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.690029 25838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:37.706707 25838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:37.707285 25838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:37.707468 25838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:37.713547 25838 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38361
I20260620 07:10:37.713564 25904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38361 every 8 connection(s)
I20260620 07:10:37.714354 25838 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:37.717170 25905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:37.717876 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25838
I20260620 07:10:37.717998 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:37.718305 8288 external_mini_cluster.cc:1428] Setting key 424c33e91deb5909f610f1608c1d55f6
I20260620 07:10:37.722720 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: Bootstrap starting.
I20260620 07:10:37.724828 25905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:37.725492 25905 log.cc:824] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:37.726580 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.719596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.8.24.62:38361)
0620 07:10:37.720170 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.720183 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.720215 (+ 32us) server_negotiation.cc:407] Connection header received
0620 07:10:37.720826 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.720841 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.721194 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.721483 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:37.721857 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.722749 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.723627 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.724064 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.724199 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.724974 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:37.724979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:37.724992 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:37.725334 (+ 342us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:37.725383 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:37.725401 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:37.725488 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.725610 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.725838 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
I20260620 07:10:37.727361 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: No bootstrap required, opened a new log
I20260620 07:10:37.730621 25905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } }
I20260620 07:10:37.730811 25905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:37.730847 25905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5313807ecc7b41aaa5662d9c11f030a4, State: Initialized, Role: FOLLOWER
I20260620 07:10:37.731189 25905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } }
I20260620 07:10:37.731297 25905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:37.731339 25905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:37.731403 25905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:37.732151 25905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } }
I20260620 07:10:37.732414 25905 leader_election.cc:302] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5313807ecc7b41aaa5662d9c11f030a4; no voters:
I20260620 07:10:37.732633 25905 leader_election.cc:288] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:37.732743 25910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:37.732962 25910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [term 1 LEADER]: Becoming Leader. State: Replica: 5313807ecc7b41aaa5662d9c11f030a4, State: Running, Role: LEADER
I20260620 07:10:37.733239 25910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } }
I20260620 07:10:37.733740 25905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:37.734249 25912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5313807ecc7b41aaa5662d9c11f030a4. Latest consensus state: current_term: 1 leader_uuid: "5313807ecc7b41aaa5662d9c11f030a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } } }
I20260620 07:10:37.734366 25912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:37.734634 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5313807ecc7b41aaa5662d9c11f030a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5313807ecc7b41aaa5662d9c11f030a4" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38361 } } }
I20260620 07:10:37.734720 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:37.735006 25917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:37.737625 25917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:37.742597 25917 catalog_manager.cc:1347] Generated new cluster ID: fba5953afe9a485fa42bf4a8ed53dd78
I20260620 07:10:37.742655 25917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:37.755705 25917 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:37.756848 25917 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:37.768106 25917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: Generated new TSK 0
I20260620 07:10:37.768886 25917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:37.771142 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--builtin_ntp_servers=127.8.24.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:37.877071 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:37.877305 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:37.877354 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:37.881270 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:37.881315 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:37.881385 25929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:37.885964 25929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:37.886931 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:37.888240 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:37.897001 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:37.897020 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:37.897104 25929 server_base.cc:1034] running on GCE node
W20260620 07:10:37.897001 25937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:37.897562 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:37.898113 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:37.899258 25929 hybrid_clock.cc:648] HybridClock initialized: now 1781939437899242 us; error 28 us; skew 500 ppm
I20260620 07:10:37.900878 25929 webserver.cc:460] Webserver started at http://127.8.24.1:41877/ using document root <none> and password file <none>
I20260620 07:10:37.901386 25929 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:37.901451 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:37.901633 25929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:37.903595 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "4da3830561ec44c68ec8e8b907e76007"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "1bc31bd9fd1d7f051ddb3b8ff470da18"
server_key_iv: "d5dfb0e669ec46cda06dc7a96966a901"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.904114 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "4da3830561ec44c68ec8e8b907e76007"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "1bc31bd9fd1d7f051ddb3b8ff470da18"
server_key_iv: "d5dfb0e669ec46cda06dc7a96966a901"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.907523 25929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 07:10:37.909723 25943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.910847 25929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:37.910946 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "4da3830561ec44c68ec8e8b907e76007"
format_stamp: "Formatted at 2026-06-20 07:10:37 on dist-test-slave-wjw3"
server_key: "1bc31bd9fd1d7f051ddb3b8ff470da18"
server_key_iv: "d5dfb0e669ec46cda06dc7a96966a901"
server_key_version: "encryptionkey@0"
I20260620 07:10:37.911046 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:37.924925 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:37.925503 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:37.925649 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:37.926146 25929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:37.926991 25929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:37.927037 25929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.927085 25929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:37.927111 25929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:37.934856 25929 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:43709
I20260620 07:10:37.934894 26056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:43709 every 8 connection(s)
I20260620 07:10:37.935746 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:37.936568 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 25929
I20260620 07:10:37.936681 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:37.936980 8288 external_mini_cluster.cc:1428] Setting key 31e931f3d737552f37f111a5de5af032
I20260620 07:10:37.938913 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--builtin_ntp_servers=127.8.24.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:37.944967 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.937549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58609 (local address 127.8.24.62:38361)
0620 07:10:37.937660 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:37.937664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:37.938800 (+ 1136us) server_negotiation.cc:407] Connection header received
0620 07:10:37.940345 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:37.940348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:37.940397 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:37.940467 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:37.941524 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.942046 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.943134 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.943351 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.943467 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.944518 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:37.944522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:37.944523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:37.944536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:37.944556 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:37.944563 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:37.944621 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:37.944752 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 07:10:37.944824 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 07:10:37.945819 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:37.937982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38361 (local address 127.8.24.1:58609)
0620 07:10:37.938675 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:37.938713 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:37.940117 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:37.940656 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:37.940665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:37.940773 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:37.941393 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.941408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.942161 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.942165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.943023 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:37.943032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:37.943511 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:37.943516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:37.943669 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:37.944275 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:37.944277 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:37.944318 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:37.944379 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:37.944629 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:37.944638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:37.944644 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:10:37.944748 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 07:10:37.945032 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
I20260620 07:10:37.947175 26057 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38361
I20260620 07:10:37.947402 26057 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:37.947928 26057 heartbeater.cc:511] Master 127.8.24.62:38361 requested a full tablet report, sending...
I20260620 07:10:37.949484 25869 ts_manager.cc:194] Registered new tserver with Master: 4da3830561ec44c68ec8e8b907e76007 (127.8.24.1:43709)
I20260620 07:10:37.950987 25869 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:58609
W20260620 07:10:38.046088 26060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:38.046321 26060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:38.046355 26060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:38.050163 26060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:38.050208 26060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:38.050258 26060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:38.054899 26060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:38.055869 26060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:38.057132 26060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:38.065891 26068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:38.065948 26066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:38.065881 26065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:38.066023 26060 server_base.cc:1034] running on GCE node
I20260620 07:10:38.066517 26060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:38.067118 26060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:38.068301 26060 hybrid_clock.cc:648] HybridClock initialized: now 1781939438068285 us; error 25 us; skew 500 ppm
I20260620 07:10:38.070073 26060 webserver.cc:460] Webserver started at http://127.8.24.2:37275/ using document root <none> and password file <none>
I20260620 07:10:38.070607 26060 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:38.070672 26060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:38.070865 26060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:38.072853 26060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "4a2355e81f30459b8d85bff8e7d47ad8"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "62f452d42a6683494be6ad8135b8af1e"
server_key_iv: "67e23540939eea53d49c79343dd0fe5f"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.073385 26060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "4a2355e81f30459b8d85bff8e7d47ad8"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "62f452d42a6683494be6ad8135b8af1e"
server_key_iv: "67e23540939eea53d49c79343dd0fe5f"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.076812 26060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:38.078960 26074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.080049 26060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:38.080153 26060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "4a2355e81f30459b8d85bff8e7d47ad8"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "62f452d42a6683494be6ad8135b8af1e"
server_key_iv: "67e23540939eea53d49c79343dd0fe5f"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.080236 26060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:38.095546 26060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:38.096122 26060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:38.096264 26060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:38.096753 26060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:38.097604 26060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:38.097651 26060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.097699 26060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:38.097724 26060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.105283 26060 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:40501
I20260620 07:10:38.105327 26187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:40501 every 8 connection(s)
I20260620 07:10:38.106119 26060 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:38.114037 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.107750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:32943 (local address 127.8.24.62:38361)
0620 07:10:38.107876 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.107880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.108846 (+ 966us) server_negotiation.cc:407] Connection header received
0620 07:10:38.110100 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.110103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.110147 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.110214 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:38.110993 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.111512 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.112522 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.112718 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.112780 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.113693 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:38.113695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:38.113697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:38.113707 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:38.113722 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:38.113728 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:38.113773 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.113870 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.113917 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:10:38.114750 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26060
I20260620 07:10:38.114810 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.108143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38361 (local address 127.8.24.2:32943)
0620 07:10:38.108735 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.108773 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.109935 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.110353 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.110359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.110436 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:38.110892 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.110904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.111597 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.111599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.112437 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.112445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.112811 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.112813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.112958 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.113479 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:38.113481 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:38.113518 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:38.113582 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:38.113782 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:38.113786 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:38.113790 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.113867 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.114062 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":98,"threads_started":1}
I20260620 07:10:38.114850 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:38.115156 8288 external_mini_cluster.cc:1428] Setting key 48de78fe004ca96361cc87ab1f928534
I20260620 07:10:38.115909 26188 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38361
I20260620 07:10:38.116114 26188 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:38.116606 26188 heartbeater.cc:511] Master 127.8.24.62:38361 requested a full tablet report, sending...
I20260620 07:10:38.116917 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--builtin_ntp_servers=127.8.24.20:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:38.117573 25869 ts_manager.cc:194] Registered new tserver with Master: 4a2355e81f30459b8d85bff8e7d47ad8 (127.8.24.2:40501)
I20260620 07:10:38.118218 25869 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:32943
W20260620 07:10:38.220661 26191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:38.220898 26191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:38.220948 26191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:38.224910 26191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:38.224957 26191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:38.225028 26191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:38.229610 26191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:38.230561 26191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:38.231902 26191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:38.241166 26197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:38.241181 26199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:38.241402 26191 server_base.cc:1034] running on GCE node
W20260620 07:10:38.241169 26196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:38.241844 26191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:38.242400 26191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:38.243505 26191 hybrid_clock.cc:648] HybridClock initialized: now 1781939438243485 us; error 30 us; skew 500 ppm
I20260620 07:10:38.245098 26191 webserver.cc:460] Webserver started at http://127.8.24.3:43871/ using document root <none> and password file <none>
I20260620 07:10:38.245608 26191 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:38.245663 26191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:38.245847 26191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:38.247757 26191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "ce46e893cf9529e0eec5960b25df5f2c"
server_key_iv: "ddf49f3098a3cca41f5d2135d3e3e56b"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.248243 26191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "ce46e893cf9529e0eec5960b25df5f2c"
server_key_iv: "ddf49f3098a3cca41f5d2135d3e3e56b"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.251533 26191 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:38.253736 26205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.254812 26191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:38.254909 26191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3"
format_stamp: "Formatted at 2026-06-20 07:10:38 on dist-test-slave-wjw3"
server_key: "ce46e893cf9529e0eec5960b25df5f2c"
server_key_iv: "ddf49f3098a3cca41f5d2135d3e3e56b"
server_key_version: "encryptionkey@0"
I20260620 07:10:38.254998 26191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:38.268916 26191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:38.269515 26191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:38.269663 26191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:38.270175 26191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:38.271049 26191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:38.271095 26191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.271145 26191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:38.271170 26191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:38.278851 26191 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:41225
I20260620 07:10:38.278875 26318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:41225 every 8 connection(s)
I20260620 07:10:38.279680 26191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:38.282639 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26191
I20260620 07:10:38.282727 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:38.282987 8288 external_mini_cluster.cc:1428] Setting key e46cc2b9e5bf03cac4efbc210ff57506
I20260620 07:10:38.288062 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.281268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:41703 (local address 127.8.24.62:38361)
0620 07:10:38.281388 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.281391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.282374 (+ 983us) server_negotiation.cc:407] Connection header received
0620 07:10:38.283994 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.283996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.284041 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.284130 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:38.285028 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.285535 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.286558 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.286757 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.286819 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.287724 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:38.287727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:38.287729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:38.287740 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:38.287759 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:38.287766 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:38.287815 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.287911 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.287957 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:10:38.288829 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.281645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38361 (local address 127.8.24.3:41703)
0620 07:10:38.282239 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.282277 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.283798 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.284280 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.284288 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.284393 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:38.284884 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.284902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.285630 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.285632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.286469 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.286478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.286850 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.286852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.286996 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.287490 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:38.287492 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:38.287540 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:38.287595 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:38.287820 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:38.287824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:38.287827 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.287906 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.288116 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":96,"threads_started":1}
I20260620 07:10:38.289774 26319 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38361
I20260620 07:10:38.289968 26319 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:38.290417 26319 heartbeater.cc:511] Master 127.8.24.62:38361 requested a full tablet report, sending...
I20260620 07:10:38.291282 25869 ts_manager.cc:194] Registered new tserver with Master: 2fb5d5b8b8b44a799aad9c0fede7b7e3 (127.8.24.3:41225)
I20260620 07:10:38.291813 25869 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:41703
I20260620 07:10:38.296279 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:38.300985 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.297936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.8.24.62:38361)
0620 07:10:38.298050 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.298054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.298198 (+ 144us) server_negotiation.cc:407] Connection header received
0620 07:10:38.298287 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.298289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.298324 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.298381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:38.298586 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.299049 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.299829 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.300030 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.300094 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.300675 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:38.300678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:38.300680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:38.300691 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:38.300708 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:38.300715 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:38.300765 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.300821 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.300864 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:38.303192 25869 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:38.305997 25869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:38.319504 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.314677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.8.24.1:43709)
0620 07:10:38.315018 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.315030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.315059 (+ 29us) server_negotiation.cc:407] Connection header received
0620 07:10:38.315124 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.315129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.315289 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.315427 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:38.315612 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316672 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318359 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.318851 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318942 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.319112 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.319220 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.319312 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
I20260620 07:10:38.319499 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.314046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43709 (local address 127.0.0.1:42844)
0620 07:10:38.314558 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.314569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.314636 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.315402 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.315405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.315422 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:38.315522 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.315528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316772 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.316776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.318264 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.318271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.318954 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.319063 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.319097 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.319195 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.319341 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":61,"thread_start_us":82,"threads_started":1}
I20260620 07:10:38.319510 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.313464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40501 (local address 127.0.0.1:54376)
0620 07:10:38.314218 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.314232 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.314352 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.315092 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.315097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.315131 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:38.315293 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.315300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316742 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.316745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.317868 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.317877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.319020 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.319024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.319118 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.319140 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.319206 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.319339 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":84,"spinlock_wait_cycles":14592,"thread_start_us":86,"threads_started":1}
I20260620 07:10:38.319511 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.314002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.8.24.2:40501)
0620 07:10:38.314490 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.314502 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.314522 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:10:38.314606 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.314611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.314810 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.314947 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:38.315421 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316617 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.317961 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.318679 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318778 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.318959 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.319255 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.319342 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":105,"threads_started":1}
I20260620 07:10:38.320219 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.313353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:41225 (local address 127.0.0.1:41280)
0620 07:10:38.314181 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.314228 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.314353 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.315091 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.315096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.315127 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:38.315263 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.315269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316291 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.316294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.318047 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.318053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.319856 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.319860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.319941 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.319964 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.320037 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.320101 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":5,"spinlock_wait_cycles":7808,"thread_start_us":69,"threads_started":1}
I20260620 07:10:38.320312 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.313489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.8.24.3:41225)
0620 07:10:38.313868 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.313880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.314303 (+ 423us) server_negotiation.cc:407] Connection header received
0620 07:10:38.314540 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.314546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.314741 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.314869 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:38.315369 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.316197 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318147 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.318850 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.318942 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.319098 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.320078 (+ 980us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.320170 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":149,"threads_started":1}
I20260620 07:10:38.322276 25991 tablet_service.cc:1467] Processing CreateTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 (DEFAULT_TABLE table=test-table [id=0f7f835d37d94e139393cea84556b741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:38.322463 26253 tablet_service.cc:1467] Processing CreateTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 (DEFAULT_TABLE table=test-table [id=0f7f835d37d94e139393cea84556b741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:38.323122 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d76de340974c0f9cdc3cfb72d02216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:38.323186 26253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d76de340974c0f9cdc3cfb72d02216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:38.325364 26122 tablet_service.cc:1467] Processing CreateTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 (DEFAULT_TABLE table=test-table [id=0f7f835d37d94e139393cea84556b741]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:38.326347 26122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d76de340974c0f9cdc3cfb72d02216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:38.328573 26336 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: Bootstrap starting.
I20260620 07:10:38.329759 26337 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Bootstrap starting.
I20260620 07:10:38.331333 26336 tablet_bootstrap.cc:654] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:38.331753 26337 tablet_bootstrap.cc:654] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:38.332190 26336 log.cc:824] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:38.332522 26337 log.cc:824] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:38.333994 26336 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: No bootstrap required, opened a new log
I20260620 07:10:38.334172 26337 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: No bootstrap required, opened a new log
I20260620 07:10:38.334180 26336 ts_tablet_manager.cc:1397] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260620 07:10:38.334335 26337 ts_tablet_manager.cc:1397] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:10:38.337383 26336 raft_consensus.cc:348] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.337442 26337 raft_consensus.cc:348] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.337591 26336 raft_consensus.cc:374] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:38.337625 26336 raft_consensus.cc:729] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da3830561ec44c68ec8e8b907e76007, State: Initialized, Role: FOLLOWER
I20260620 07:10:38.337625 26337 raft_consensus.cc:374] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:38.337680 26337 raft_consensus.cc:729] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fb5d5b8b8b44a799aad9c0fede7b7e3, State: Initialized, Role: FOLLOWER
I20260620 07:10:38.338013 26336 consensus_queue.cc:260] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.338040 26337 consensus_queue.cc:260] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.338593 26319 heartbeater.cc:503] Master 127.8.24.62:38361 was elected leader, sending a full tablet report...
I20260620 07:10:38.338595 26057 heartbeater.cc:503] Master 127.8.24.62:38361 was elected leader, sending a full tablet report...
I20260620 07:10:38.338937 26336 ts_tablet_manager.cc:1428] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260620 07:10:38.338935 26337 ts_tablet_manager.cc:1428] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:10:38.332877 26338 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Bootstrap starting.
I20260620 07:10:38.341223 26338 tablet_bootstrap.cc:654] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:38.342264 26338 log.cc:824] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:38.344054 26338 tablet_bootstrap.cc:492] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: No bootstrap required, opened a new log
I20260620 07:10:38.344198 26338 ts_tablet_manager.cc:1397] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260620 07:10:38.347034 26338 raft_consensus.cc:348] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.347231 26338 raft_consensus.cc:374] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:38.347283 26338 raft_consensus.cc:729] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2355e81f30459b8d85bff8e7d47ad8, State: Initialized, Role: FOLLOWER
I20260620 07:10:38.347688 26338 consensus_queue.cc:260] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.348256 26188 heartbeater.cc:503] Master 127.8.24.62:38361 was elected leader, sending a full tablet report...
I20260620 07:10:38.348522 26338 ts_tablet_manager.cc:1428] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 07:10:38.348591 26341 raft_consensus.cc:482] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:38.348731 26341 raft_consensus.cc:504] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.349664 26341 leader_election.cc:288] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fb5d5b8b8b44a799aad9c0fede7b7e3 (127.8.24.3:41225), 4a2355e81f30459b8d85bff8e7d47ad8 (127.8.24.2:40501)
I20260620 07:10:38.353776 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.350241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:34867 (local address 127.8.24.3:41225)
0620 07:10:38.350338 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.350342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.350354 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:10:38.350391 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.350394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.350428 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.350497 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:38.350723 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.351405 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.352789 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.353336 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.353430 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.353498 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.353606 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.353672 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:10:38.353780 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.349812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:41225 (local address 127.8.24.1:34867)
0620 07:10:38.350021 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.350036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.350109 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.350513 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.350516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.350532 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:38.350635 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.350641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.351520 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.351524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.352684 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.352694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.353441 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.353444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.353513 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.353532 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.353596 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.353663 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260620 07:10:38.353941 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.349873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:40501 (local address 127.8.24.1:45637)
0620 07:10:38.350200 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.350210 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:38.350295 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:38.350701 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:38.350703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:38.350719 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:38.350844 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.350850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.351726 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.351729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.352843 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:38.352850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.353611 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.353614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:38.353693 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.353715 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:10:38.353777 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:10:38.353833 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":82,"threads_started":1}
I20260620 07:10:38.353983 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.349983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:45637 (local address 127.8.24.2:40501)
0620 07:10:38.350081 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.350086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.350276 (+ 190us) server_negotiation.cc:407] Connection header received
0620 07:10:38.350479 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.350482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.350524 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.350596 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:38.350957 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.351645 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.352942 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.353499 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.353586 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.353631 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.353800 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.353866 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:10:38.354408 26273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d76de340974c0f9cdc3cfb72d02216" candidate_uuid: "4da3830561ec44c68ec8e8b907e76007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" is_pre_election: true
I20260620 07:10:38.354617 26273 raft_consensus.cc:2454] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4da3830561ec44c68ec8e8b907e76007 in term 0.
I20260620 07:10:38.354982 25946 leader_election.cc:302] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fb5d5b8b8b44a799aad9c0fede7b7e3, 4da3830561ec44c68ec8e8b907e76007; no voters:
I20260620 07:10:38.355021 26142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d76de340974c0f9cdc3cfb72d02216" candidate_uuid: "4da3830561ec44c68ec8e8b907e76007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" is_pre_election: true
I20260620 07:10:38.355217 26142 raft_consensus.cc:2454] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4da3830561ec44c68ec8e8b907e76007 in term 0.
I20260620 07:10:38.355219 26341 raft_consensus.cc:2781] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:38.355273 26341 raft_consensus.cc:482] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:38.355301 26341 raft_consensus.cc:3037] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:38.356343 26341 raft_consensus.cc:504] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.356628 26341 leader_election.cc:288] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [CANDIDATE]: Term 1 election: Requested vote from peers 2fb5d5b8b8b44a799aad9c0fede7b7e3 (127.8.24.3:41225), 4a2355e81f30459b8d85bff8e7d47ad8 (127.8.24.2:40501)
I20260620 07:10:38.356884 26142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d76de340974c0f9cdc3cfb72d02216" candidate_uuid: "4da3830561ec44c68ec8e8b907e76007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a2355e81f30459b8d85bff8e7d47ad8"
I20260620 07:10:38.356884 26273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d76de340974c0f9cdc3cfb72d02216" candidate_uuid: "4da3830561ec44c68ec8e8b907e76007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3"
W20260620 07:10:38.356909 26189 tablet.cc:2369] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:38.356988 26273 raft_consensus.cc:3037] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:38.357013 26142 raft_consensus.cc:3037] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:38.358105 26142 raft_consensus.cc:2454] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da3830561ec44c68ec8e8b907e76007 in term 1.
I20260620 07:10:38.358124 26273 raft_consensus.cc:2454] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da3830561ec44c68ec8e8b907e76007 in term 1.
I20260620 07:10:38.358359 25947 leader_election.cc:302] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a2355e81f30459b8d85bff8e7d47ad8, 4da3830561ec44c68ec8e8b907e76007; no voters:
I20260620 07:10:38.358537 26341 raft_consensus.cc:2781] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:38.358718 26341 raft_consensus.cc:686] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [term 1 LEADER]: Becoming Leader. State: Replica: 4da3830561ec44c68ec8e8b907e76007, State: Running, Role: LEADER
I20260620 07:10:38.358985 26341 consensus_queue.cc:237] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } }
I20260620 07:10:38.361527 25869 catalog_manager.cc:5495] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4da3830561ec44c68ec8e8b907e76007 (127.8.24.1). New cstate: current_term: 1 leader_uuid: "4da3830561ec44c68ec8e8b907e76007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4da3830561ec44c68ec8e8b907e76007" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 43709 } health_report { overall_health: HEALTHY } } }
I20260620 07:10:38.372835 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.369642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.8.24.1:43709)
0620 07:10:38.369744 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:38.369749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:38.369761 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:10:38.369899 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:38.369902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:38.369955 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:38.370035 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:38.370258 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.370739 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.371554 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:38.371776 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:38.371857 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:38.372131 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:38.372484 (+ 353us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:38.372581 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:38.372658 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 07:10:38.372705 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:10:38.377678 26142 raft_consensus.cc:1261] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 1 FOLLOWER]: Refusing update from remote peer 4da3830561ec44c68ec8e8b907e76007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:38.377678 26273 raft_consensus.cc:1261] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 1 FOLLOWER]: Refusing update from remote peer 4da3830561ec44c68ec8e8b907e76007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:38.378353 26341 consensus_queue.cc:1035] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a2355e81f30459b8d85bff8e7d47ad8" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:38.378561 26346 consensus_queue.cc:1035] T 32d76de340974c0f9cdc3cfb72d02216 P 4da3830561ec44c68ec8e8b907e76007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fb5d5b8b8b44a799aad9c0fede7b7e3" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:38.388125 26351 mvcc.cc:205] Tried to move back new op lower bound from 7298823939589115904 to 7298823939520745472. Current Snapshot: MvccSnapshot[applied={T|T < 7298823939589115904 or (T in {7298823939589115904})}]
I20260620 07:10:38.394349 25867 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:38.394500 25867 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:38.396660 25867 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5313807ecc7b41aaa5662d9c11f030a4: Sending DeleteTablet for 3 replicas of tablet 32d76de340974c0f9cdc3cfb72d02216
I20260620 07:10:38.397339 26253 tablet_service.cc:1514] Processing DeleteTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:38 UTC) from {username='slave'} at 127.0.0.1:41280
I20260620 07:10:38.397429 26122 tablet_service.cc:1514] Processing DeleteTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:38 UTC) from {username='slave'} at 127.0.0.1:54376
I20260620 07:10:38.397576 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet 32d76de340974c0f9cdc3cfb72d02216 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:38 UTC) from {username='slave'} at 127.0.0.1:42844
I20260620 07:10:38.398211 26360 tablet_replica.cc:330] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: stopping tablet replica
I20260620 07:10:38.398273 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25929
I20260620 07:10:38.398495 26360 raft_consensus.cc:2229] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:38.398757 26360 raft_consensus.cc:2258] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:38.398972 26359 tablet_replica.cc:330] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: stopping tablet replica
I20260620 07:10:38.399224 26359 raft_consensus.cc:2229] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:38.399487 26359 raft_consensus.cc:2258] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:38.400607 26360 ts_tablet_manager.cc:1905] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:38.400852 26359 ts_tablet_manager.cc:1905] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:38.402599 26360 ts_tablet_manager.cc:1918] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:38.402678 26360 log.cc:1196] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/32d76de340974c0f9cdc3cfb72d02216
I20260620 07:10:38.402925 26360 ts_tablet_manager.cc:1939] T 32d76de340974c0f9cdc3cfb72d02216 P 4a2355e81f30459b8d85bff8e7d47ad8: Deleting consensus metadata
I20260620 07:10:38.403267 26359 ts_tablet_manager.cc:1918] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:38.403352 26359 log.cc:1196] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/32d76de340974c0f9cdc3cfb72d02216
I20260620 07:10:38.403666 26359 ts_tablet_manager.cc:1939] T 32d76de340974c0f9cdc3cfb72d02216 P 2fb5d5b8b8b44a799aad9c0fede7b7e3: Deleting consensus metadata
I20260620 07:10:38.403945 25856 catalog_manager.cc:4841] TS 4a2355e81f30459b8d85bff8e7d47ad8 (127.8.24.2:40501): tablet 32d76de340974c0f9cdc3cfb72d02216 (table test-table [id=0f7f835d37d94e139393cea84556b741]) successfully deleted
I20260620 07:10:38.404637 25855 catalog_manager.cc:4841] TS 2fb5d5b8b8b44a799aad9c0fede7b7e3 (127.8.24.3:41225): tablet 32d76de340974c0f9cdc3cfb72d02216 (table test-table [id=0f7f835d37d94e139393cea84556b741]) successfully deleted
W20260620 07:10:38.405113 25854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:43709 (error 108)
I20260620 07:10:38.405752 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26060
I20260620 07:10:38.406009 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:38.405746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:43709 (local address 127.0.0.1:42874)
0620 07:10:38.405873 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:38.405937 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 07:10:38.407605 25854 catalog_manager.cc:4584] TS 4da3830561ec44c68ec8e8b907e76007 (127.8.24.1:43709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32d76de340974c0f9cdc3cfb72d02216: Network error: Client connection negotiation failed: client connection to 127.8.24.1:43709: connect: Connection refused (error 111)
I20260620 07:10:38.410921 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26191
I20260620 07:10:38.414976 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 25838
2026-06-20T07:10:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (888 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:38 dist-test-slave-wjw3 krb5kdc[26365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:38 dist-test-slave-wjw3 krb5kdc[26365](info): set up 2 sockets
Jun 20 07:10:38 dist-test-slave-wjw3 krb5kdc[26365](info): commencing operation
krb5kdc: starting...
W20260620 07:10:39.370992 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:39 dist-test-slave-wjw3 krb5kdc[26365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:10:39.475960 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38949
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:38949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:39.582598 26378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:39.582907 26378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:39.582976 26378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:39.587040 26378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:39.587090 26378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:39.587108 26378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:39.587123 26378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:39.587141 26378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:39.592134 26378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:38949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:38949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:39.593011 26378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:39.593887 26378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:39.601423 26383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:39.601542 26378 server_base.cc:1034] running on GCE node
W20260620 07:10:39.601440 26386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:39.601440 26384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:39.602146 26378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:39.603091 26378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:39.604238 26378 hybrid_clock.cc:648] HybridClock initialized: now 1781939439604228 us; error 23 us; skew 500 ppm
Jun 20 07:10:39 dist-test-slave-wjw3 krb5kdc[26365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:39.606673 26378 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:10:39.607458 26378 webserver.cc:460] Webserver started at http://127.8.24.62:32815/ using document root <none> and password file <none>
I20260620 07:10:39.607929 26378 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:39.607982 26378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:39.608134 26378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:39.609956 26378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "b6be88e0376e4da0a9c5761c237b1be5"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "61fbb0315929b5b77af59ee20dc79551"
server_key_iv: "162ff4b4b6fb1123ec78b0b7102a0c52"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.610477 26378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "b6be88e0376e4da0a9c5761c237b1be5"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "61fbb0315929b5b77af59ee20dc79551"
server_key_iv: "162ff4b4b6fb1123ec78b0b7102a0c52"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.613813 26378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 07:10:39.615984 26393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:39.617077 26378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:39.617177 26378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "b6be88e0376e4da0a9c5761c237b1be5"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "61fbb0315929b5b77af59ee20dc79551"
server_key_iv: "162ff4b4b6fb1123ec78b0b7102a0c52"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.617267 26378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:39.631107 26378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:39.633967 26378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:39.634123 26378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:39.640544 26378 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:38949
I20260620 07:10:39.640568 26445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:38949 every 8 connection(s)
I20260620 07:10:39.641383 26378 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:39.643975 26446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:39.649016 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: Bootstrap starting.
I20260620 07:10:39.650911 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26378
I20260620 07:10:39.651062 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:39.651336 8288 external_mini_cluster.cc:1428] Setting key 4bd19a1b73039f9d50dfb4c827edbf7b
I20260620 07:10:39.651464 26446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:39.652279 26446 log.cc:824] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:39.654039 26446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: No bootstrap required, opened a new log
I20260620 07:10:39.657888 26446 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } }
I20260620 07:10:39.658159 26446 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:39.658202 26446 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6be88e0376e4da0a9c5761c237b1be5, State: Initialized, Role: FOLLOWER
I20260620 07:10:39.658697 26446 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } }
I20260620 07:10:39.658854 26446 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:39.658954 26446 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:39.659057 26446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 07:10:39 dist-test-slave-wjw3 krb5kdc[26365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:39.660143 26446 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } }
I20260620 07:10:39.660490 26446 leader_election.cc:302] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6be88e0376e4da0a9c5761c237b1be5; no voters:
I20260620 07:10:39.660820 26446 leader_election.cc:288] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:39.660955 26451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:39.661175 26451 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [term 1 LEADER]: Becoming Leader. State: Replica: b6be88e0376e4da0a9c5761c237b1be5, State: Running, Role: LEADER
I20260620 07:10:39.661497 26451 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } }
I20260620 07:10:39.662122 26446 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:39.662698 26451 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6be88e0376e4da0a9c5761c237b1be5. Latest consensus state: current_term: 1 leader_uuid: "b6be88e0376e4da0a9c5761c237b1be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } } }
I20260620 07:10:39.662839 26451 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:39.662698 26453 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6be88e0376e4da0a9c5761c237b1be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6be88e0376e4da0a9c5761c237b1be5" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 38949 } } }
I20260620 07:10:39.662973 26453 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:39.663189 26458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:39.666232 26458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:39.666353 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:39.652437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.8.24.62:38949)
0620 07:10:39.652842 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:39.652856 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:39.652927 (+ 71us) server_negotiation.cc:407] Connection header received
0620 07:10:39.653549 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:39.653565 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:39.653868 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:39.654187 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:39.654996 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.656055 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.657236 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.657701 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.657822 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:39.660669 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:39.660675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:39.660691 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:39.660720 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:39.662675 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:39.663195 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:39.663202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:39.663208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:39.663339 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:39.663682 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:39.663686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:39.663687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:39.664205 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:39.665024 (+ 819us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:39.665186 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 07:10:39.665461 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"spinlock_wait_cycles":23424,"thread_start_us":131,"threads_started":1}
I20260620 07:10:39.671635 26458 catalog_manager.cc:1347] Generated new cluster ID: 714c8362c8484e71a53df2c1b352b727
I20260620 07:10:39.671695 26458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:39.684911 26458 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:39.685756 26458 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:39.691468 26458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: Generated new TSK 0
I20260620 07:10:39.692153 26458 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:10:39.720428 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:39.824589 26472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:39.824816 26472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:39.824870 26472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:39.828708 26472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:39.828763 26472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:39.828840 26472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:39.833849 26472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:39.834786 26472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:39.835752 26472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:39.844475 26480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:39.844478 26477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:39.844478 26478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:39.844740 26472 server_base.cc:1034] running on GCE node
I20260620 07:10:39.845160 26472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:39.845732 26472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:39.846884 26472 hybrid_clock.cc:648] HybridClock initialized: now 1781939439846871 us; error 29 us; skew 500 ppm
Jun 20 07:10:39 dist-test-slave-wjw3 krb5kdc[26365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:39.849330 26472 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:10:39.850126 26472 webserver.cc:460] Webserver started at http://127.8.24.1:46399/ using document root <none> and password file <none>
I20260620 07:10:39.850622 26472 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:39.850684 26472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:39.850852 26472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:39.852789 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "4838249b698a4dcbb217a980630b93c0"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "2885cf60ef62d5c450d8680aab8b379f"
server_key_iv: "2b703fa3be736fd006944214fced5b87"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.853312 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "4838249b698a4dcbb217a980630b93c0"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "2885cf60ef62d5c450d8680aab8b379f"
server_key_iv: "2b703fa3be736fd006944214fced5b87"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.856653 26472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:39.858737 26487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:39.859829 26472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:39.859925 26472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "4838249b698a4dcbb217a980630b93c0"
format_stamp: "Formatted at 2026-06-20 07:10:39 on dist-test-slave-wjw3"
server_key: "2885cf60ef62d5c450d8680aab8b379f"
server_key_iv: "2b703fa3be736fd006944214fced5b87"
server_key_version: "encryptionkey@0"
I20260620 07:10:39.860013 26472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:39.886726 26472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:39.889379 26472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:39.889547 26472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:39.890094 26472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:39.890851 26472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:39.890897 26472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:39.890946 26472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:39.890974 26472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:39.898825 26472 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:37501
I20260620 07:10:39.898869 26600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:37501 every 8 connection(s)
I20260620 07:10:39.899783 26472 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:39.905525 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26472
I20260620 07:10:39.905642 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:39.905891 8288 external_mini_cluster.cc:1428] Setting key 02afe54ac548ffee7af2422081a11db5
Jun 20 07:10:39 dist-test-slave-wjw3 krb5kdc[26365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:39.911957 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:39.901350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:48155 (local address 127.8.24.62:38949)
0620 07:10:39.901463 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:39.901467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:39.902215 (+ 748us) server_negotiation.cc:407] Connection header received
0620 07:10:39.903047 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:39.903051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:39.903105 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:39.903188 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:39.904314 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.904840 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.905973 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.906206 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.906303 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:39.908745 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:39.908748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:39.908750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:39.908773 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:39.909971 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:39.910485 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:39.910488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:39.910489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:39.910571 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:39.910914 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:39.910916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:39.910917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:39.911075 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:39.911142 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:39.911746 (+ 604us) server_negotiation.cc:299] Negotiation successful
0620 07:10:39.911843 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:10:39.913086 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:39.901677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38949 (local address 127.8.24.1:48155)
0620 07:10:39.902082 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:39.902122 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:39.902858 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:39.903353 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:39.903360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:39.903720 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:39.904177 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:39.904191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.904952 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.904955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:39.905857 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:39.905867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:39.906350 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:39.906354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:39.906558 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:39.907060 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:39.907075 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:39.908583 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:39.910079 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:39.910085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:39.910099 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:39.910405 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:39.910657 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:39.910661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:39.910663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:39.910836 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:39.911163 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:39.911169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:39.911493 (+ 324us) client_negotiation.cc:769] Sending connection context
0620 07:10:39.911716 (+ 223us) client_negotiation.cc:240] Negotiation successful
0620 07:10:39.912045 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
I20260620 07:10:39.914165 26601 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38949
I20260620 07:10:39.914376 26601 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:10:39.914816 26601 heartbeater.cc:511] Master 127.8.24.62:38949 requested a full tablet report, sending...
I20260620 07:10:39.916254 26410 ts_manager.cc:194] Registered new tserver with Master: 4838249b698a4dcbb217a980630b93c0 (127.8.24.1:37501)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:10:39.917359 26410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:48155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:10:39.928385 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:40.030970 26606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:40.031204 26606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:40.031247 26606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:40.035120 26606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:40.035168 26606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:40.035236 26606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:40.040194 26606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:40.041121 26606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:40.042049 26606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:40.050810 26614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:40.050817 26611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.051009 26606 server_base.cc:1034] running on GCE node
W20260620 07:10:40.050825 26612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.051505 26606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:40.052112 26606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:40.053284 26606 hybrid_clock.cc:648] HybridClock initialized: now 1781939440053261 us; error 35 us; skew 500 ppm
Jun 20 07:10:40 dist-test-slave-wjw3 krb5kdc[26365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:40.055760 26606 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:10:40.056578 26606 webserver.cc:460] Webserver started at http://127.8.24.2:46357/ using document root <none> and password file <none>
I20260620 07:10:40.057104 26606 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:40.057161 26606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:40.057341 26606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:40.059293 26606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "d671705d39c74a22a2926abb9184877b"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "bcba74fbd5f2a1bd4d40936ff37e9b7a"
server_key_iv: "8d9783bb25d1bf03d1fa52502c832f4e"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.059847 26606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "d671705d39c74a22a2926abb9184877b"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "bcba74fbd5f2a1bd4d40936ff37e9b7a"
server_key_iv: "8d9783bb25d1bf03d1fa52502c832f4e"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.063364 26606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:40.065646 26621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.066756 26606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 07:10:40.066847 26606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "d671705d39c74a22a2926abb9184877b"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "bcba74fbd5f2a1bd4d40936ff37e9b7a"
server_key_iv: "8d9783bb25d1bf03d1fa52502c832f4e"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.066936 26606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:40.093880 26606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:40.096434 26606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:40.096606 26606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:40.097136 26606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:40.097886 26606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:40.097929 26606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.097970 26606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:40.097999 26606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.105891 26606 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:39251
I20260620 07:10:40.105918 26734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:39251 every 8 connection(s)
I20260620 07:10:40.106696 26606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:40.113112 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26606
I20260620 07:10:40.113260 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:40.113521 8288 external_mini_cluster.cc:1428] Setting key 96905ed1ffd88b97676ab945d954b150
Jun 20 07:10:40 dist-test-slave-wjw3 krb5kdc[26365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:40.119091 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.108330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:59183 (local address 127.8.24.62:38949)
0620 07:10:40.108450 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.108453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.109226 (+ 773us) server_negotiation.cc:407] Connection header received
0620 07:10:40.110387 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.110390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.110433 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.110501 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:40.111559 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.112071 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.113256 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.113501 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.113599 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.116004 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:40.116007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:40.116009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:40.116031 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:40.117320 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.117730 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.117733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.117735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.117816 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.118067 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.118069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.118071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.118270 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:40.118360 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.118818 (+ 458us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.118939 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:10:40.119764 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.108683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38949 (local address 127.8.24.2:59183)
0620 07:10:40.109088 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.109128 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.110215 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.110651 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.110658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.110980 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:40.111415 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.111426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.112158 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.112161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.113138 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.113148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.113641 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.113646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.113828 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.114339 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:40.114354 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:40.115856 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:40.117406 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:40.117410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:40.117420 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:40.117652 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:40.117888 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:40.117890 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:40.117891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:40.117999 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:40.118349 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:40.118353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:40.118590 (+ 237us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.118737 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.118958 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":116,"threads_started":1}
I20260620 07:10:40.120867 26735 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38949
I20260620 07:10:40.121066 26735 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:40.121497 26735 heartbeater.cc:511] Master 127.8.24.62:38949 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
I20260620 07:10:40.122433 26410 ts_manager.cc:194] Registered new tserver with Master: d671705d39c74a22a2926abb9184877b (127.8.24.2:39251)
I20260620 07:10:40.122954 26410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:59183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:10:40.138075 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:40.241976 26740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:40.242219 26740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:40.242286 26740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:40.246197 26740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:40.246253 26740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:40.246330 26740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:40.251343 26740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:38949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:40.252288 26740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:40.253239 26740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:40.261876 26745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:40.261896 26746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.262130 26740 server_base.cc:1034] running on GCE node
W20260620 07:10:40.261898 26748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.262619 26740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:40.263273 26740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:40.264427 26740 hybrid_clock.cc:648] HybridClock initialized: now 1781939440264402 us; error 40 us; skew 500 ppm
Jun 20 07:10:40 dist-test-slave-wjw3 krb5kdc[26365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:40.269815 26740 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:10:40.270656 26740 webserver.cc:460] Webserver started at http://127.8.24.3:37159/ using document root <none> and password file <none>
I20260620 07:10:40.271178 26740 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:40.271234 26740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:40.271418 26740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:40.273391 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "becd9ce103a0488cb774c3fd764cf9d8"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "32a6646555b73246bf676633da34d1f5"
server_key_iv: "68123a0f074481692dff7af8465e45ab"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.273891 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "becd9ce103a0488cb774c3fd764cf9d8"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "32a6646555b73246bf676633da34d1f5"
server_key_iv: "68123a0f074481692dff7af8465e45ab"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.277273 26740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:10:40.279484 26755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.280603 26740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:40.280694 26740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "becd9ce103a0488cb774c3fd764cf9d8"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "32a6646555b73246bf676633da34d1f5"
server_key_iv: "68123a0f074481692dff7af8465e45ab"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.280783 26740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:40.298638 26740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:40.301182 26740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:40.301335 26740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:40.301852 26740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:40.302616 26740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:40.302660 26740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.302700 26740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:40.302729 26740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.310449 26740 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:42617
I20260620 07:10:40.310473 26868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:42617 every 8 connection(s)
I20260620 07:10:40.311228 26740 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:40.312762 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26740
I20260620 07:10:40.312865 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:40.313138 8288 external_mini_cluster.cc:1428] Setting key 188c4e4f7f9d186c954d4c19f01efbdf
Jun 20 07:10:40 dist-test-slave-wjw3 krb5kdc[26365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:40.323324 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.312902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:46989 (local address 127.8.24.62:38949)
0620 07:10:40.313027 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.313032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.314033 (+ 1001us) server_negotiation.cc:407] Connection header received
0620 07:10:40.315404 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.315406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.315475 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.315546 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:40.316556 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.317056 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.318054 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.318247 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.318308 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.320490 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:40.320494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:40.320496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:40.320520 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:40.321835 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.322247 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.322249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.322250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.322316 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.322547 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.322549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.322550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.322695 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:40.322759 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.323135 (+ 376us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.323218 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:40.324043 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.313377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:38949 (local address 127.8.24.3:46989)
0620 07:10:40.313836 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.313889 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.315244 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.315672 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.315678 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.315999 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:40.316451 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.316463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.317140 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.317143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.317972 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.317980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.318339 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.318342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.318469 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.318972 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:40.318987 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:40.320365 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:40.321927 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:40.321932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:40.321942 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:40.322174 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:40.322371 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:40.322373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:40.322374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:40.322483 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:40.322759 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:40.322763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:40.322955 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.323105 (+ 150us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.323317 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":127,"threads_started":1}
I20260620 07:10:40.324952 26869 heartbeater.cc:347] Connected to a master server at 127.8.24.62:38949
I20260620 07:10:40.325146 26869 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:40.325553 26869 heartbeater.cc:511] Master 127.8.24.62:38949 requested a full tablet report, sending...
I20260620 07:10:40.326470 26409 ts_manager.cc:194] Registered new tserver with Master: becd9ce103a0488cb774c3fd764cf9d8 (127.8.24.3:42617)
I20260620 07:10:40.327014 26409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:46989
I20260620 07:10:40.337879 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:40.345762 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.339602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.8.24.62:38949)
0620 07:10:40.339727 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.339731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.339859 (+ 128us) server_negotiation.cc:407] Connection header received
0620 07:10:40.339945 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.339947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.339981 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.340048 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:40.340578 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.341051 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.341748 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.341934 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.341993 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.343309 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:40.343312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:40.343313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:40.343328 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:40.344386 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.344781 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.344784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.344785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.344847 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:40.345107 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:40.345109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:40.345110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:40.345271 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:40.345372 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.345553 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.345620 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 07:10:40.348294 26409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:40.351070 26409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:40.364300 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.358693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37501 (local address 127.0.0.1:46392)
0620 07:10:40.358975 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.359016 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.359128 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.359847 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.359853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.359876 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:40.360000 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.360007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.361317 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.361320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.362745 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.362756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.363709 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.363713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.363828 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.363862 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.364019 (+ 157us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.364134 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260620 07:10:40.364306 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.358833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.8.24.1:37501)
0620 07:10:40.359103 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.359115 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.359140 (+ 25us) server_negotiation.cc:407] Connection header received
0620 07:10:40.359262 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.359267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.359507 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.359661 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:40.360121 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.361195 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.362853 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.363610 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.363709 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.363858 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.364050 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.364175 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260620 07:10:40.364907 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.359366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:39251 (local address 127.0.0.1:36030)
0620 07:10:40.359985 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.359999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.360082 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.360894 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.360897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.360914 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:40.361007 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.361012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.362343 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.362346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.363528 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.363535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.364537 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.364541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.364638 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.364660 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.364726 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.364784 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":68,"thread_start_us":112,"threads_started":1}
I20260620 07:10:40.365083 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.359688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:42617 (local address 127.0.0.1:58420)
0620 07:10:40.360248 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.360260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.360340 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.360699 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.360702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.360718 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:40.360798 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.360803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.361880 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.361883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.363488 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.363499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.364733 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.364736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.364800 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.364818 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.364879 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.364946 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":19,"thread_start_us":69,"threads_started":1}
I20260620 07:10:40.365142 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.360046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.8.24.3:42617)
0620 07:10:40.360358 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.360369 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.360402 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:10:40.360464 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.360467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.360594 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.360709 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:40.360880 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.361765 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.363631 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.364435 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.364546 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.364660 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.364919 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.365007 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
I20260620 07:10:40.365283 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.359567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.8.24.2:39251)
0620 07:10:40.360474 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.360485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.360505 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:10:40.360574 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.360579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.360752 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.361139 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:40.361285 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.362207 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.363638 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.364435 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.364536 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.364660 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.365053 (+ 393us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.365148 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":79,"threads_started":1}
I20260620 07:10:40.367558 26535 tablet_service.cc:1467] Processing CreateTablet for tablet 26df5b808f7d43cd8198c08c5364b990 (DEFAULT_TABLE table=test-table [id=ec4a8d35e635441a8fa7980fd1444fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:40.367799 26803 tablet_service.cc:1467] Processing CreateTablet for tablet 26df5b808f7d43cd8198c08c5364b990 (DEFAULT_TABLE table=test-table [id=ec4a8d35e635441a8fa7980fd1444fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:40.368468 26535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df5b808f7d43cd8198c08c5364b990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:40.368673 26803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df5b808f7d43cd8198c08c5364b990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:40.374202 26887 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Bootstrap starting.
I20260620 07:10:40.374920 26886 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Bootstrap starting.
I20260620 07:10:40.376399 26887 tablet_bootstrap.cc:654] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:40.376581 26886 tablet_bootstrap.cc:654] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:40.377465 26886 log.cc:824] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:40.377127 26669 tablet_service.cc:1467] Processing CreateTablet for tablet 26df5b808f7d43cd8198c08c5364b990 (DEFAULT_TABLE table=test-table [id=ec4a8d35e635441a8fa7980fd1444fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:40.377558 26887 log.cc:824] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:40.378100 26669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df5b808f7d43cd8198c08c5364b990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:40.379089 26887 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: No bootstrap required, opened a new log
I20260620 07:10:40.379253 26887 ts_tablet_manager.cc:1397] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:10:40.379279 26886 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: No bootstrap required, opened a new log
I20260620 07:10:40.379482 26886 ts_tablet_manager.cc:1397] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 07:10:40.382304 26887 raft_consensus.cc:348] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.382508 26887 raft_consensus.cc:374] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:40.382567 26887 raft_consensus.cc:729] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: becd9ce103a0488cb774c3fd764cf9d8, State: Initialized, Role: FOLLOWER
I20260620 07:10:40.382942 26887 consensus_queue.cc:260] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.383046 26886 raft_consensus.cc:348] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.383291 26886 raft_consensus.cc:374] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:40.383352 26886 raft_consensus.cc:729] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4838249b698a4dcbb217a980630b93c0, State: Initialized, Role: FOLLOWER
I20260620 07:10:40.383684 26869 heartbeater.cc:503] Master 127.8.24.62:38949 was elected leader, sending a full tablet report...
I20260620 07:10:40.383819 26886 consensus_queue.cc:260] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.383970 26887 ts_tablet_manager.cc:1428] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:40.384575 26601 heartbeater.cc:503] Master 127.8.24.62:38949 was elected leader, sending a full tablet report...
I20260620 07:10:40.384680 26886 ts_tablet_manager.cc:1428] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:40.385227 26892 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Bootstrap starting.
I20260620 07:10:40.387163 26892 tablet_bootstrap.cc:654] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:40.388393 26892 log.cc:824] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:40.389948 26892 tablet_bootstrap.cc:492] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: No bootstrap required, opened a new log
I20260620 07:10:40.390100 26892 ts_tablet_manager.cc:1397] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 07:10:40.392782 26892 raft_consensus.cc:348] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.393018 26892 raft_consensus.cc:374] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:40.393057 26892 raft_consensus.cc:729] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d671705d39c74a22a2926abb9184877b, State: Initialized, Role: FOLLOWER
I20260620 07:10:40.393396 26892 consensus_queue.cc:260] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.393898 26735 heartbeater.cc:503] Master 127.8.24.62:38949 was elected leader, sending a full tablet report...
I20260620 07:10:40.394100 26892 ts_tablet_manager.cc:1428] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
W20260620 07:10:40.400728 26602 tablet.cc:2369] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:10:40.562150 26870 tablet.cc:2369] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:40.606040 26894 raft_consensus.cc:482] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:40.606249 26894 raft_consensus.cc:504] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.607366 26894 leader_election.cc:288] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4838249b698a4dcbb217a980630b93c0 (127.8.24.1:37501), becd9ce103a0488cb774c3fd764cf9d8 (127.8.24.3:42617)
W20260620 07:10:40.607717 26736 tablet.cc:2369] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:40.610846 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.607631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37501 (local address 127.8.24.2:48263)
0620 07:10:40.607768 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.607784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.607883 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.608210 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.608213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.608232 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:40.608340 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.608346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.609068 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.609072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.609840 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.609848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.610458 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.610462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.610565 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.610582 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.610644 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.610715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 07:10:40.610846 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.607751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48263 (local address 127.8.24.1:37501)
0620 07:10:40.607878 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.607882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.607899 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:10:40.607967 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.607970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.608018 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.608093 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:40.608449 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.608963 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.609958 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.610352 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.610442 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.610502 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.610669 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.610741 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:10:40.611157 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.608260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:48889 (local address 127.8.24.3:42617)
0620 07:10:40.608381 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.608385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.608396 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:10:40.608437 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.608439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.608475 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.608535 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:40.608795 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.609273 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.610260 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.610644 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.610733 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.610788 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.610975 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.611052 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:10:40.611501 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.607904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:42617 (local address 127.8.24.2:48889)
0620 07:10:40.608190 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.608203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:40.608284 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:40.608571 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:40.608574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:40.608588 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:40.608691 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.608696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.609373 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.609377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.610142 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:40.610150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.610748 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.610751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:40.610843 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.610861 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:10:40.611287 (+ 426us) client_negotiation.cc:240] Negotiation successful
0620 07:10:40.611360 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":67,"threads_started":1}
I20260620 07:10:40.611507 26555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df5b808f7d43cd8198c08c5364b990" candidate_uuid: "d671705d39c74a22a2926abb9184877b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4838249b698a4dcbb217a980630b93c0" is_pre_election: true
I20260620 07:10:40.611810 26555 raft_consensus.cc:2454] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d671705d39c74a22a2926abb9184877b in term 0.
I20260620 07:10:40.612105 26823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df5b808f7d43cd8198c08c5364b990" candidate_uuid: "d671705d39c74a22a2926abb9184877b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becd9ce103a0488cb774c3fd764cf9d8" is_pre_election: true
I20260620 07:10:40.612293 26622 leader_election.cc:302] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4838249b698a4dcbb217a980630b93c0, d671705d39c74a22a2926abb9184877b; no voters:
I20260620 07:10:40.612357 26823 raft_consensus.cc:2454] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d671705d39c74a22a2926abb9184877b in term 0.
I20260620 07:10:40.612532 26894 raft_consensus.cc:2781] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:40.612622 26894 raft_consensus.cc:482] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:40.612670 26894 raft_consensus.cc:3037] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:40.613641 26894 raft_consensus.cc:504] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.613933 26894 leader_election.cc:288] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [CANDIDATE]: Term 1 election: Requested vote from peers 4838249b698a4dcbb217a980630b93c0 (127.8.24.1:37501), becd9ce103a0488cb774c3fd764cf9d8 (127.8.24.3:42617)
I20260620 07:10:40.614262 26555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df5b808f7d43cd8198c08c5364b990" candidate_uuid: "d671705d39c74a22a2926abb9184877b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4838249b698a4dcbb217a980630b93c0"
I20260620 07:10:40.614270 26823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df5b808f7d43cd8198c08c5364b990" candidate_uuid: "d671705d39c74a22a2926abb9184877b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becd9ce103a0488cb774c3fd764cf9d8"
I20260620 07:10:40.614377 26823 raft_consensus.cc:3037] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:40.614368 26555 raft_consensus.cc:3037] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:40.615293 26555 raft_consensus.cc:2454] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d671705d39c74a22a2926abb9184877b in term 1.
I20260620 07:10:40.615341 26823 raft_consensus.cc:2454] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d671705d39c74a22a2926abb9184877b in term 1.
I20260620 07:10:40.615641 26622 leader_election.cc:302] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: becd9ce103a0488cb774c3fd764cf9d8, d671705d39c74a22a2926abb9184877b; no voters:
I20260620 07:10:40.615787 26894 raft_consensus.cc:2781] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:40.615972 26894 raft_consensus.cc:686] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 1 LEADER]: Becoming Leader. State: Replica: d671705d39c74a22a2926abb9184877b, State: Running, Role: LEADER
I20260620 07:10:40.616287 26894 consensus_queue.cc:237] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } }
I20260620 07:10:40.619107 26409 catalog_manager.cc:5495] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b reported cstate change: term changed from 0 to 1, leader changed from <none> to d671705d39c74a22a2926abb9184877b (127.8.24.2). New cstate: current_term: 1 leader_uuid: "d671705d39c74a22a2926abb9184877b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d671705d39c74a22a2926abb9184877b" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 } health_report { overall_health: UNKNOWN } } }
I20260620 07:10:40.653280 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.649560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.8.24.2:39251)
0620 07:10:40.649682 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.649686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.649702 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:10:40.649800 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.649803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.649862 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.649962 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:40.650166 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.650905 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.651746 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.652086 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.652175 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.652424 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:40.652897 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:40.653030 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.653094 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.653129 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:10:40.658744 26823 raft_consensus.cc:1261] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 1 FOLLOWER]: Refusing update from remote peer d671705d39c74a22a2926abb9184877b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:40.658746 26555 raft_consensus.cc:1261] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0 [term 1 FOLLOWER]: Refusing update from remote peer d671705d39c74a22a2926abb9184877b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:40.659401 26894 consensus_queue.cc:1035] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [LEADER]: Connected to new peer: Peer: permanent_uuid: "becd9ce103a0488cb774c3fd764cf9d8" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:40.659608 26896 consensus_queue.cc:1035] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4838249b698a4dcbb217a980630b93c0" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 37501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:40.668630 26901 mvcc.cc:205] Tried to move back new op lower bound from 7298823948931723264 to 7298823948766658560. Current Snapshot: MvccSnapshot[applied={T|T < 7298823948931723264}]
I20260620 07:10:40.675374 26410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:40.675570 26410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:40.677804 26410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6be88e0376e4da0a9c5761c237b1be5: Sending DeleteTablet for 3 replicas of tablet 26df5b808f7d43cd8198c08c5364b990
I20260620 07:10:40.678459 26535 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df5b808f7d43cd8198c08c5364b990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:46392
I20260620 07:10:40.678575 26803 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df5b808f7d43cd8198c08c5364b990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:58420
I20260620 07:10:40.678901 26909 tablet_replica.cc:330] T 26df5b808f7d43cd8198c08c5364b990 P 4838249b698a4dcbb217a980630b93c0: stopping tablet replica
I20260620 07:10:40.679042 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26472
I20260620 07:10:40.679181 26669 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df5b808f7d43cd8198c08c5364b990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:40 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:36030
I20260620 07:10:40.679689 26911 tablet_replica.cc:330] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: stopping tablet replica
I20260620 07:10:40.679908 26911 raft_consensus.cc:2229] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:10:40.680181 26911 raft_consensus.cc:2258] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:40.680491 26910 tablet_replica.cc:330] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: stopping tablet replica
I20260620 07:10:40.680745 26910 raft_consensus.cc:2229] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:40.681016 26910 raft_consensus.cc:2258] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:40.681849 26911 ts_tablet_manager.cc:1905] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:40.682268 26910 ts_tablet_manager.cc:1905] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:40.684546 26911 ts_tablet_manager.cc:1918] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:40.684697 26911 log.cc:1196] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/26df5b808f7d43cd8198c08c5364b990
I20260620 07:10:40.684733 26910 ts_tablet_manager.cc:1918] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:40.684820 26910 log.cc:1196] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/26df5b808f7d43cd8198c08c5364b990
I20260620 07:10:40.685137 26911 ts_tablet_manager.cc:1939] T 26df5b808f7d43cd8198c08c5364b990 P d671705d39c74a22a2926abb9184877b: Deleting consensus metadata
I20260620 07:10:40.685137 26910 ts_tablet_manager.cc:1939] T 26df5b808f7d43cd8198c08c5364b990 P becd9ce103a0488cb774c3fd764cf9d8: Deleting consensus metadata
I20260620 07:10:40.686156 26394 catalog_manager.cc:4841] TS d671705d39c74a22a2926abb9184877b (127.8.24.2:39251): tablet 26df5b808f7d43cd8198c08c5364b990 (table test-table [id=ec4a8d35e635441a8fa7980fd1444fbc]) successfully deleted
I20260620 07:10:40.686314 26394 catalog_manager.cc:4841] TS becd9ce103a0488cb774c3fd764cf9d8 (127.8.24.3:42617): tablet 26df5b808f7d43cd8198c08c5364b990 (table test-table [id=ec4a8d35e635441a8fa7980fd1444fbc]) successfully deleted
W20260620 07:10:40.688968 26394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:37501 (error 108)
I20260620 07:10:40.689702 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26606
I20260620 07:10:40.689858 26884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.689569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:37501 (local address 127.0.0.1:46400)
0620 07:10:40.689722 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:40.689784 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260620 07:10:40.690719 26394 catalog_manager.cc:4584] TS 4838249b698a4dcbb217a980630b93c0 (127.8.24.1:37501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26df5b808f7d43cd8198c08c5364b990: Network error: Client connection negotiation failed: client connection to 127.8.24.1:37501: connect: Connection refused (error 111)
I20260620 07:10:40.694353 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26740
I20260620 07:10:40.698437 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26378
2026-06-20T07:10:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2282 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T07:10:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:40Z Disabled control of system clock
I20260620 07:10:40.724745 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44765
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:40.827209 26916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:40.827426 26916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:40.827481 26916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:40.831319 26916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:40.831365 26916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:40.831378 26916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:40.831398 26916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:40.831413 26916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:40.835980 26916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:40.836836 26916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:40.838028 26916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:40.845809 26924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.845849 26916 server_base.cc:1034] running on GCE node
W20260620 07:10:40.845809 26921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:40.845831 26922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:40.846737 26916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:40.847769 26916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:40.848912 26916 hybrid_clock.cc:648] HybridClock initialized: now 1781939440848888 us; error 35 us; skew 500 ppm
I20260620 07:10:40.850553 26916 webserver.cc:460] Webserver started at http://127.8.24.62:41697/ using document root <none> and password file <none>
I20260620 07:10:40.851055 26916 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:40.851111 26916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:40.851295 26916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:40.853219 26916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "97d71b15ac154c4d967ae24e517ae602"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "cc3f82112ee3e64fecbc4a4d128b0196"
server_key_iv: "a1dac08b36e95aee6a75c381c9e9fae6"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.853729 26916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "97d71b15ac154c4d967ae24e517ae602"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "cc3f82112ee3e64fecbc4a4d128b0196"
server_key_iv: "a1dac08b36e95aee6a75c381c9e9fae6"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.857054 26916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 07:10:40.859212 26930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:40.860339 26916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:10:40.860442 26916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "97d71b15ac154c4d967ae24e517ae602"
format_stamp: "Formatted at 2026-06-20 07:10:40 on dist-test-slave-wjw3"
server_key: "cc3f82112ee3e64fecbc4a4d128b0196"
server_key_iv: "a1dac08b36e95aee6a75c381c9e9fae6"
server_key_version: "encryptionkey@0"
I20260620 07:10:40.860527 26916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:40.875093 26916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:40.875767 26916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:40.875921 26916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:40.882171 26916 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44765
I20260620 07:10:40.882187 26982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44765 every 8 connection(s)
I20260620 07:10:40.883021 26916 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:40.885921 26983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:40.889271 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 26916
I20260620 07:10:40.889379 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:40.889628 8288 external_mini_cluster.cc:1428] Setting key e615a83b04c9cc65c696606738a12bbc
I20260620 07:10:40.891453 26983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: Bootstrap starting.
I20260620 07:10:40.893489 26983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:40.894160 26983 log.cc:824] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:40.895841 26983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: No bootstrap required, opened a new log
I20260620 07:10:40.898285 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:40.890955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.8.24.62:44765)
0620 07:10:40.891618 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:40.891634 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:40.891669 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:10:40.892265 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:40.892281 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:40.892653 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:40.892953 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:40.893358 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.894351 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.895180 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:40.895686 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:40.895825 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:40.896612 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:40.896616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:40.896628 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:40.896979 (+ 351us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:40.897031 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:40.897053 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:40.897134 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:40.897262 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 07:10:40.897486 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260620 07:10:40.898783 26983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } }
I20260620 07:10:40.898983 26983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:40.899037 26983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d71b15ac154c4d967ae24e517ae602, State: Initialized, Role: FOLLOWER
I20260620 07:10:40.899430 26983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } }
I20260620 07:10:40.899590 26983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:40.899650 26983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:40.899744 26983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:40.900593 26983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } }
I20260620 07:10:40.900882 26983 leader_election.cc:302] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97d71b15ac154c4d967ae24e517ae602; no voters:
I20260620 07:10:40.901126 26983 leader_election.cc:288] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:40.901204 26988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:40.901389 26988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [term 1 LEADER]: Becoming Leader. State: Replica: 97d71b15ac154c4d967ae24e517ae602, State: Running, Role: LEADER
I20260620 07:10:40.901676 26988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } }
I20260620 07:10:40.902163 26983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:40.903187 26990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97d71b15ac154c4d967ae24e517ae602. Latest consensus state: current_term: 1 leader_uuid: "97d71b15ac154c4d967ae24e517ae602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } } }
I20260620 07:10:40.903196 26989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97d71b15ac154c4d967ae24e517ae602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d71b15ac154c4d967ae24e517ae602" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44765 } } }
I20260620 07:10:40.903402 26989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:40.903349 26990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:40.904452 26997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:40.907351 26997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:40.912346 26997 catalog_manager.cc:1347] Generated new cluster ID: 211d39e617a244988bda01742259d6e9
I20260620 07:10:40.912412 26997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:40.928992 26997 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:40.929801 26997 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:40.936081 26997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: Generated new TSK 0
I20260620 07:10:40.936694 26997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:40.939855 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--builtin_ntp_servers=127.8.24.20:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:41.045049 27007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:41.045277 27007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:41.045318 27007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:41.049203 27007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:41.049253 27007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:41.049319 27007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:41.053933 27007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:41.054960 27007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:41.056298 27007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:41.065425 27015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:41.065426 27013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:41.065408 27012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:41.065634 27007 server_base.cc:1034] running on GCE node
I20260620 07:10:41.066013 27007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:41.066576 27007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:41.067718 27007 hybrid_clock.cc:648] HybridClock initialized: now 1781939441067704 us; error 28 us; skew 500 ppm
I20260620 07:10:41.069330 27007 webserver.cc:460] Webserver started at http://127.8.24.1:33565/ using document root <none> and password file <none>
I20260620 07:10:41.069834 27007 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:41.069886 27007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:41.070045 27007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:41.071991 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "68aa1e6896b643f38ee624d9333c0c65"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "097d1a297d85338a78566d8ae4d729ac"
server_key_iv: "66c6861e56a39dd88def72b0d3995109"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.072469 27007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "68aa1e6896b643f38ee624d9333c0c65"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "097d1a297d85338a78566d8ae4d729ac"
server_key_iv: "66c6861e56a39dd88def72b0d3995109"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.075839 27007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:41.077958 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.079041 27007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:41.079128 27007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "68aa1e6896b643f38ee624d9333c0c65"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "097d1a297d85338a78566d8ae4d729ac"
server_key_iv: "66c6861e56a39dd88def72b0d3995109"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.079206 27007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:41.096985 27007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:41.097563 27007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:41.097695 27007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:41.098188 27007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:41.099032 27007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:41.099076 27007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.099112 27007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:41.099130 27007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.107192 27007 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:46483
I20260620 07:10:41.107242 27134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:46483 every 8 connection(s)
I20260620 07:10:41.108018 27007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:41.115648 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27007
I20260620 07:10:41.115746 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:41.115971 8288 external_mini_cluster.cc:1428] Setting key 2357300357af19a0527c47a0cefd0386
I20260620 07:10:41.116791 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.109614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:58879 (local address 127.8.24.62:44765)
0620 07:10:41.109735 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.109739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.110760 (+ 1021us) server_negotiation.cc:407] Connection header received
0620 07:10:41.112083 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.112085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.112128 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.112196 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:41.112992 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.113511 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.114525 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.114737 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.114805 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.116372 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:41.116375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:41.116377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:41.116388 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:41.116408 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:41.116417 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:41.116490 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.116623 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.116675 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:10:41.117678 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.110009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44765 (local address 127.8.24.1:58879)
0620 07:10:41.110622 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.110659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.111908 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.112339 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.112345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.112425 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:41.112890 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.112902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.113592 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.113594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.114439 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.114446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.114839 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.114842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.114973 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.116129 (+ 1156us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:41.116131 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:41.116168 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:41.116225 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:41.116499 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:41.116505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:41.116510 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.116615 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.116866 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
I20260620 07:10:41.117758 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--builtin_ntp_servers=127.8.24.20:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:41.118749 27135 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44765
I20260620 07:10:41.118964 27135 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:41.119552 27135 heartbeater.cc:511] Master 127.8.24.62:44765 requested a full tablet report, sending...
I20260620 07:10:41.120944 26947 ts_manager.cc:194] Registered new tserver with Master: 68aa1e6896b643f38ee624d9333c0c65 (127.8.24.1:46483)
I20260620 07:10:41.122048 26947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:58879
W20260620 07:10:41.220914 27138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:41.221158 27138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:41.221206 27138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:41.225111 27138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:41.225160 27138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:41.225229 27138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:41.229775 27138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:41.230726 27138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:41.232024 27138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:41.240805 27144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:41.240820 27143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:41.240813 27146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:41.240960 27138 server_base.cc:1034] running on GCE node
I20260620 07:10:41.241359 27138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:41.241928 27138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:41.243084 27138 hybrid_clock.cc:648] HybridClock initialized: now 1781939441243067 us; error 27 us; skew 500 ppm
I20260620 07:10:41.244690 27138 webserver.cc:460] Webserver started at http://127.8.24.2:36267/ using document root <none> and password file <none>
I20260620 07:10:41.245205 27138 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:41.245266 27138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:41.245440 27138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:41.247319 27138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "6e8e34c553a9bb248ed3f003d23340ec"
server_key_iv: "0878b29e8fa1ee183ff9093327a4b93a"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.247857 27138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "6e8e34c553a9bb248ed3f003d23340ec"
server_key_iv: "0878b29e8fa1ee183ff9093327a4b93a"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.251103 27138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:41.253285 27152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.254339 27138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:41.254436 27138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "6e8e34c553a9bb248ed3f003d23340ec"
server_key_iv: "0878b29e8fa1ee183ff9093327a4b93a"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.254518 27138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:41.276986 27138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:41.277601 27138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:41.277755 27138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:41.278285 27138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:41.279148 27138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:41.279192 27138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.279243 27138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:41.279271 27138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.287197 27138 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:34811
I20260620 07:10:41.287243 27265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:34811 every 8 connection(s)
I20260620 07:10:41.288003 27138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:41.293287 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27138
I20260620 07:10:41.293392 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:41.293697 8288 external_mini_cluster.cc:1428] Setting key 44a41eef7983910ea4f9da29f8196ac6
I20260620 07:10:41.295687 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--builtin_ntp_servers=127.8.24.20:34827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:41.296723 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.289613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:51817 (local address 127.8.24.62:44765)
0620 07:10:41.289725 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.289728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.290707 (+ 979us) server_negotiation.cc:407] Connection header received
0620 07:10:41.291987 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.291990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.292044 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.292113 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:41.292926 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.293472 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.294821 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.295086 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.295159 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.296333 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:41.296336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:41.296338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:41.296350 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:41.296370 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:41.296377 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:41.296430 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.296569 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.296617 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:10:41.297663 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.289965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44765 (local address 127.8.24.2:51817)
0620 07:10:41.290569 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.290606 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.291819 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.292256 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.292262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.292343 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:41.292811 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.292824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.293596 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.293600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.294715 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.294725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.295211 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.295215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.295397 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.296056 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:41.296058 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:41.296112 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:41.296191 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:41.296444 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:41.296451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:41.296456 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.296575 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.296851 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
I20260620 07:10:41.298770 27266 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44765
I20260620 07:10:41.298974 27266 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:41.299496 27266 heartbeater.cc:511] Master 127.8.24.62:44765 requested a full tablet report, sending...
I20260620 07:10:41.300508 26947 ts_manager.cc:194] Registered new tserver with Master: ad33fcfc0bc5421c8c7a6f3c3e95eccd (127.8.24.2:34811)
I20260620 07:10:41.301242 26947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:51817
W20260620 07:10:41.400141 27269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:41.400357 27269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:41.400391 27269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:41.404327 27269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:41.404373 27269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:41.404422 27269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:41.408859 27269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:34827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:44765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:41.409725 27269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:41.410920 27269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:41.419538 27277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:41.419548 27274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:41.419584 27269 server_base.cc:1034] running on GCE node
W20260620 07:10:41.419538 27275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:41.420056 27269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:41.420652 27269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:41.421809 27269 hybrid_clock.cc:648] HybridClock initialized: now 1781939441421789 us; error 32 us; skew 500 ppm
I20260620 07:10:41.423485 27269 webserver.cc:460] Webserver started at http://127.8.24.3:43497/ using document root <none> and password file <none>
I20260620 07:10:41.424002 27269 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:41.424063 27269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:41.424237 27269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:41.426101 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "a45ca2ea8d8d42909484553df32d5e14"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "c504757ecb5e2fc4be78d46c7865b833"
server_key_iv: "57ad1ee66663db6ca175fd5644f18e31"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.426604 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "a45ca2ea8d8d42909484553df32d5e14"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "c504757ecb5e2fc4be78d46c7865b833"
server_key_iv: "57ad1ee66663db6ca175fd5644f18e31"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.429940 27269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260620 07:10:41.432140 27283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.433209 27269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:10:41.433306 27269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "a45ca2ea8d8d42909484553df32d5e14"
format_stamp: "Formatted at 2026-06-20 07:10:41 on dist-test-slave-wjw3"
server_key: "c504757ecb5e2fc4be78d46c7865b833"
server_key_iv: "57ad1ee66663db6ca175fd5644f18e31"
server_key_version: "encryptionkey@0"
I20260620 07:10:41.433393 27269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:41.445549 27269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:41.446080 27269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:41.446272 27269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:41.446810 27269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:41.447677 27269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:41.447722 27269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.447770 27269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:41.447794 27269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:41.455384 27269 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:44147
I20260620 07:10:41.455410 27396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:44147 every 8 connection(s)
I20260620 07:10:41.456166 27269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:41.461166 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27269
I20260620 07:10:41.461259 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:41.461490 8288 external_mini_cluster.cc:1428] Setting key ef2e5f54e17405ee9452fe46524f9219
I20260620 07:10:41.464762 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.457750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:34825 (local address 127.8.24.62:44765)
0620 07:10:41.457893 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.457897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.458955 (+ 1058us) server_negotiation.cc:407] Connection header received
0620 07:10:41.460279 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.460282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.460338 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.460410 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:41.461265 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.461834 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.463049 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.463356 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.463496 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.464414 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:41.464417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:41.464419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:41.464430 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:41.464448 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:41.464455 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:41.464508 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.464615 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.464659 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:10:41.465566 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.458181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:44765 (local address 127.8.24.3:34825)
0620 07:10:41.458815 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.458852 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.460078 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.460579 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.460585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.460667 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:41.461123 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.461136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.461965 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.461968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.462938 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.462948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.463515 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.463519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.463674 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.464196 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:41.464197 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:41.464234 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:41.464292 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:41.464515 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:41.464520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:41.464524 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.464611 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.464822 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260620 07:10:41.466557 27397 heartbeater.cc:347] Connected to a master server at 127.8.24.62:44765
I20260620 07:10:41.466751 27397 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:41.467187 27397 heartbeater.cc:511] Master 127.8.24.62:44765 requested a full tablet report, sending...
I20260620 07:10:41.468106 26947 ts_manager.cc:194] Registered new tserver with Master: a45ca2ea8d8d42909484553df32d5e14 (127.8.24.3:44147)
I20260620 07:10:41.468621 26947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:34825
I20260620 07:10:41.474859 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:41.479811 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.476647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.8.24.62:44765)
0620 07:10:41.476767 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.476772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.476891 (+ 119us) server_negotiation.cc:407] Connection header received
0620 07:10:41.476985 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.476987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.477023 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.477085 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:41.477280 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.477762 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.478489 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.478693 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.478758 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.479396 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:41.479399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:41.479401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:41.479412 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:41.479430 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:41.479466 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:41.479573 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.479649 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.479679 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:10:41.482206 26947 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:41.485179 26947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:41.498075 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.492646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46483 (local address 127.0.0.1:59438)
0620 07:10:41.493079 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.493121 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.493236 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.494213 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.494219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.494239 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:41.494373 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.494379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495509 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.495513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.496745 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.496755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.497563 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.497567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.497677 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.497715 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.497821 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.497935 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":62,"threads_started":1}
I20260620 07:10:41.498090 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.492772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.8.24.1:46483)
0620 07:10:41.493747 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.493758 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.493782 (+ 24us) server_negotiation.cc:407] Connection header received
0620 07:10:41.493854 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.493860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.494062 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.494515 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:41.494657 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495404 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.496877 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.497446 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.497539 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.497730 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.497850 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.497942 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":133,"threads_started":1}
I20260620 07:10:41.498247 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.492747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:44147 (local address 127.0.0.1:34522)
0620 07:10:41.493432 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.493444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.493511 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.494365 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.494367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.494381 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:41.494463 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.494470 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495826 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.495829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.496934 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.496942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.497878 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.497882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.497967 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.497989 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.498049 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.498116 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260620 07:10:41.498391 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.493298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:34811 (local address 127.0.0.1:57942)
0620 07:10:41.494009 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.494020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.494171 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.494585 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.494588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.494602 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:41.494671 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.494676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495894 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.495897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.496855 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.496865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.498072 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.498075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.498149 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.498170 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.498230 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.498288 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260620 07:10:41.498771 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.493494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.8.24.2:34811)
0620 07:10:41.493888 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.493898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.494105 (+ 207us) server_negotiation.cc:407] Connection header received
0620 07:10:41.494303 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.494309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.494472 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.494618 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:41.494742 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495810 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.496971 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.497747 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.497848 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.498017 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.498518 (+ 501us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.498624 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":112,"threads_started":1}
I20260620 07:10:41.498840 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.493510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34522 (local address 127.8.24.3:44147)
0620 07:10:41.493936 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.493947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.493970 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:10:41.494054 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.494059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.494237 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.494390 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:41.494546 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.495718 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.497020 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.497747 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.498409 (+ 662us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.498584 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.498624 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.498700 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":133,"threads_started":1}
I20260620 07:10:41.501228 27331 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0e9e93ca44070a7480862f193328c (DEFAULT_TABLE table=test-table [id=eded0d050e994db5bee83ced6dbdea4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:41.501389 27200 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0e9e93ca44070a7480862f193328c (DEFAULT_TABLE table=test-table [id=eded0d050e994db5bee83ced6dbdea4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:41.501389 27069 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0e9e93ca44070a7480862f193328c (DEFAULT_TABLE table=test-table [id=eded0d050e994db5bee83ced6dbdea4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:41.502170 27331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0e9e93ca44070a7480862f193328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:41.502357 27069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0e9e93ca44070a7480862f193328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:41.502357 27200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0e9e93ca44070a7480862f193328c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:41.508620 27414 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Bootstrap starting.
I20260620 07:10:41.509068 27415 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Bootstrap starting.
I20260620 07:10:41.509961 27416 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Bootstrap starting.
I20260620 07:10:41.510819 27414 tablet_bootstrap.cc:654] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:41.511058 27415 tablet_bootstrap.cc:654] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:41.511700 27414 log.cc:824] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:41.511749 27416 tablet_bootstrap.cc:654] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:41.511754 27415 log.cc:824] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:41.512490 27416 log.cc:824] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:41.513664 27414 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: No bootstrap required, opened a new log
I20260620 07:10:41.513664 27415 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: No bootstrap required, opened a new log
I20260620 07:10:41.513845 27415 ts_tablet_manager.cc:1397] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:10:41.513851 27414 ts_tablet_manager.cc:1397] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:41.514222 27416 tablet_bootstrap.cc:492] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: No bootstrap required, opened a new log
I20260620 07:10:41.514396 27416 ts_tablet_manager.cc:1397] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 07:10:41.517374 27416 raft_consensus.cc:348] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.517593 27416 raft_consensus.cc:374] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:41.517669 27416 raft_consensus.cc:729] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68aa1e6896b643f38ee624d9333c0c65, State: Initialized, Role: FOLLOWER
I20260620 07:10:41.518105 27416 consensus_queue.cc:260] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.517992 27414 raft_consensus.cc:348] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.517992 27415 raft_consensus.cc:348] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.518226 27414 raft_consensus.cc:374] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:41.518225 27415 raft_consensus.cc:374] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:41.518280 27415 raft_consensus.cc:729] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad33fcfc0bc5421c8c7a6f3c3e95eccd, State: Initialized, Role: FOLLOWER
I20260620 07:10:41.518308 27414 raft_consensus.cc:729] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45ca2ea8d8d42909484553df32d5e14, State: Initialized, Role: FOLLOWER
I20260620 07:10:41.518705 27135 heartbeater.cc:503] Master 127.8.24.62:44765 was elected leader, sending a full tablet report...
I20260620 07:10:41.518759 27414 consensus_queue.cc:260] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.519022 27416 ts_tablet_manager.cc:1428] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:41.519369 27397 heartbeater.cc:503] Master 127.8.24.62:44765 was elected leader, sending a full tablet report...
I20260620 07:10:41.519327 27415 consensus_queue.cc:260] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.519680 27414 ts_tablet_manager.cc:1428] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 07:10:41.520153 27266 heartbeater.cc:503] Master 127.8.24.62:44765 was elected leader, sending a full tablet report...
I20260620 07:10:41.520193 27415 ts_tablet_manager.cc:1428] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
W20260620 07:10:41.538800 27267 tablet.cc:2369] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:41.588106 27422 raft_consensus.cc:482] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:41.588281 27422 raft_consensus.cc:504] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.589355 27422 leader_election.cc:288] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68aa1e6896b643f38ee624d9333c0c65 (127.8.24.1:46483), a45ca2ea8d8d42909484553df32d5e14 (127.8.24.3:44147)
I20260620 07:10:41.592741 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.589538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46483 (local address 127.8.24.2:45035)
0620 07:10:41.589721 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.589735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.589819 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.590047 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.590049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.590064 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:41.590166 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.590172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.590848 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.590851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.591785 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.591795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.592365 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.592368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.592461 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.592482 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.592545 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.592613 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":33}
I20260620 07:10:41.592741 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.589648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:45035 (local address 127.8.24.1:46483)
0620 07:10:41.589759 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.589763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.589776 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:10:41.589919 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.589922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.589964 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.590021 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:41.590267 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.590755 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.591892 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.592267 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.592354 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.592411 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.592573 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.592638 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 07:10:41.592905 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.589581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:44147 (local address 127.8.24.2:37813)
0620 07:10:41.589929 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.589939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:41.590029 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:41.590270 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:41.590273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:41.590287 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:41.590400 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.590405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.591082 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.591085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.591939 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:41.591948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.592521 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.592524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:41.592612 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.592630 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:10:41.592691 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 07:10:41.592752 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":34,"thread_start_us":112,"threads_started":1}
I20260620 07:10:41.592897 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.589690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:37813 (local address 127.8.24.3:44147)
0620 07:10:41.589808 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.589812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.590022 (+ 210us) server_negotiation.cc:407] Connection header received
0620 07:10:41.590145 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.590147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.590187 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.590245 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:41.590504 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.590976 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.592050 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.592410 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.592497 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.592565 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.592717 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.592783 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:10:41.593473 27351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0e9e93ca44070a7480862f193328c" candidate_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45ca2ea8d8d42909484553df32d5e14" is_pre_election: true
I20260620 07:10:41.593518 27089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0e9e93ca44070a7480862f193328c" candidate_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68aa1e6896b643f38ee624d9333c0c65" is_pre_election: true
I20260620 07:10:41.593751 27351 raft_consensus.cc:2454] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad33fcfc0bc5421c8c7a6f3c3e95eccd in term 0.
I20260620 07:10:41.593773 27089 raft_consensus.cc:2454] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad33fcfc0bc5421c8c7a6f3c3e95eccd in term 0.
I20260620 07:10:41.594177 27154 leader_election.cc:302] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68aa1e6896b643f38ee624d9333c0c65, ad33fcfc0bc5421c8c7a6f3c3e95eccd; no voters:
I20260620 07:10:41.594412 27422 raft_consensus.cc:2781] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:41.594483 27422 raft_consensus.cc:482] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:41.594512 27422 raft_consensus.cc:3037] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:41.595479 27422 raft_consensus.cc:504] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.595796 27422 leader_election.cc:288] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [CANDIDATE]: Term 1 election: Requested vote from peers 68aa1e6896b643f38ee624d9333c0c65 (127.8.24.1:46483), a45ca2ea8d8d42909484553df32d5e14 (127.8.24.3:44147)
I20260620 07:10:41.596081 27089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0e9e93ca44070a7480862f193328c" candidate_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68aa1e6896b643f38ee624d9333c0c65"
I20260620 07:10:41.596110 27351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0e9e93ca44070a7480862f193328c" candidate_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45ca2ea8d8d42909484553df32d5e14"
I20260620 07:10:41.596199 27089 raft_consensus.cc:3037] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:41.596202 27351 raft_consensus.cc:3037] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:41.597363 27351 raft_consensus.cc:2454] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad33fcfc0bc5421c8c7a6f3c3e95eccd in term 1.
I20260620 07:10:41.597363 27089 raft_consensus.cc:2454] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad33fcfc0bc5421c8c7a6f3c3e95eccd in term 1.
I20260620 07:10:41.597643 27155 leader_election.cc:302] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45ca2ea8d8d42909484553df32d5e14, ad33fcfc0bc5421c8c7a6f3c3e95eccd; no voters:
I20260620 07:10:41.597784 27422 raft_consensus.cc:2781] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:41.597998 27422 raft_consensus.cc:686] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 1 LEADER]: Becoming Leader. State: Replica: ad33fcfc0bc5421c8c7a6f3c3e95eccd, State: Running, Role: LEADER
I20260620 07:10:41.598286 27422 consensus_queue.cc:237] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } }
I20260620 07:10:41.601122 26947 catalog_manager.cc:5495] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd reported cstate change: term changed from 0 to 1, leader changed from <none> to ad33fcfc0bc5421c8c7a6f3c3e95eccd (127.8.24.2). New cstate: current_term: 1 leader_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad33fcfc0bc5421c8c7a6f3c3e95eccd" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 34811 } health_report { overall_health: HEALTHY } } }
W20260620 07:10:41.609014 27136 tablet.cc:2369] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:41.618242 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.615193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.8.24.2:34811)
0620 07:10:41.615295 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:41.615299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:41.615310 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:10:41.615416 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:41.615419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:41.615499 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:41.615582 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:41.615771 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.616249 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.617052 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:41.617249 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:41.617314 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:41.617545 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:41.617912 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:41.618011 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:41.618070 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 07:10:41.618100 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:10:41.623575 27351 raft_consensus.cc:1261] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 1 FOLLOWER]: Refusing update from remote peer ad33fcfc0bc5421c8c7a6f3c3e95eccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:41.623575 27089 raft_consensus.cc:1261] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 1 FOLLOWER]: Refusing update from remote peer ad33fcfc0bc5421c8c7a6f3c3e95eccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:41.624279 27424 consensus_queue.cc:1035] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "68aa1e6896b643f38ee624d9333c0c65" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 46483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:41.624434 27422 consensus_queue.cc:1035] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45ca2ea8d8d42909484553df32d5e14" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:41.631808 27429 mvcc.cc:205] Tried to move back new op lower bound from 7298823952883568640 to 7298823952788930560. Current Snapshot: MvccSnapshot[applied={T|T < 7298823952883568640}]
I20260620 07:10:41.634364 27428 mvcc.cc:205] Tried to move back new op lower bound from 7298823952883568640 to 7298823952788930560. Current Snapshot: MvccSnapshot[applied={T|T < 7298823952883568640}]
I20260620 07:10:41.639056 26945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:41.639218 26945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:41.641467 26945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97d71b15ac154c4d967ae24e517ae602: Sending DeleteTablet for 3 replicas of tablet d9c0e9e93ca44070a7480862f193328c
I20260620 07:10:41.642150 27069 tablet_service.cc:1514] Processing DeleteTablet for tablet d9c0e9e93ca44070a7480862f193328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:41 UTC) from {username='slave'} at 127.0.0.1:59438
I20260620 07:10:41.642280 27200 tablet_service.cc:1514] Processing DeleteTablet for tablet d9c0e9e93ca44070a7480862f193328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:41 UTC) from {username='slave'} at 127.0.0.1:57942
I20260620 07:10:41.642599 27331 tablet_service.cc:1514] Processing DeleteTablet for tablet d9c0e9e93ca44070a7480862f193328c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:41 UTC) from {username='slave'} at 127.0.0.1:34522
I20260620 07:10:41.642827 27437 tablet_replica.cc:330] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65: stopping tablet replica
I20260620 07:10:41.643003 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27007
I20260620 07:10:41.643110 27437 raft_consensus.cc:2229] T d9c0e9e93ca44070a7480862f193328c P 68aa1e6896b643f38ee624d9333c0c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:41.644491 27439 tablet_replica.cc:330] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: stopping tablet replica
I20260620 07:10:41.644753 27439 raft_consensus.cc:2229] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:41.645021 27439 raft_consensus.cc:2258] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:41.646317 27439 ts_tablet_manager.cc:1905] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:41.648926 27439 ts_tablet_manager.cc:1918] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:41.649081 27439 log.cc:1196] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/d9c0e9e93ca44070a7480862f193328c
I20260620 07:10:41.649080 27438 tablet_replica.cc:330] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: stopping tablet replica
I20260620 07:10:41.649335 27438 raft_consensus.cc:2229] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:10:41.649463 27439 ts_tablet_manager.cc:1939] T d9c0e9e93ca44070a7480862f193328c P a45ca2ea8d8d42909484553df32d5e14: Deleting consensus metadata
I20260620 07:10:41.649806 27438 raft_consensus.cc:2258] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:10:41.650146 26932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:46483 (error 108)
I20260620 07:10:41.650629 26933 catalog_manager.cc:4841] TS a45ca2ea8d8d42909484553df32d5e14 (127.8.24.3:44147): tablet d9c0e9e93ca44070a7480862f193328c (table test-table [id=eded0d050e994db5bee83ced6dbdea4a]) successfully deleted
I20260620 07:10:41.651274 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.651024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:46483 (local address 127.0.0.1:59440)
0620 07:10:41.651140 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.651199 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 07:10:41.651309 27438 ts_tablet_manager.cc:1905] T d9c0e9e93ca44070a7480862f193328c P ad33fcfc0bc5421c8c7a6f3c3e95eccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:41.651394 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27138
W20260620 07:10:41.652100 26932 catalog_manager.cc:4584] TS 68aa1e6896b643f38ee624d9333c0c65 (127.8.24.1:46483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9c0e9e93ca44070a7480862f193328c: Network error: Client connection negotiation failed: client connection to 127.8.24.1:46483: connect: Connection refused (error 111)
I20260620 07:10:41.656461 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27269
I20260620 07:10:41.657454 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:41.656641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:34811 (local address 127.0.0.1:57956)
0620 07:10:41.657329 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:41.657381 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:34811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606}
I20260620 07:10:41.661002 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 26916
2026-06-20T07:10:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (962 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:41 dist-test-slave-wjw3 krb5kdc[27443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:41 dist-test-slave-wjw3 krb5kdc[27443](info): set up 2 sockets
Jun 20 07:10:41 dist-test-slave-wjw3 krb5kdc[27443](info): commencing operation
krb5kdc: starting...
W20260620 07:10:42.599093 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:42 dist-test-slave-wjw3 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:10:42.705973 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35419
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:42.809746 27456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:42.810005 27456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:42.810046 27456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:42.814014 27456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:42.814064 27456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:42.814081 27456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:42.814108 27456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:42.814134 27456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:42.819499 27456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:42.820482 27456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:42.821453 27456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:42.829145 27461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:42.829154 27464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:42.829305 27456 server_base.cc:1034] running on GCE node
W20260620 07:10:42.829178 27462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:42.830189 27456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:42.831183 27456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:42.832326 27456 hybrid_clock.cc:648] HybridClock initialized: now 1781939442832302 us; error 36 us; skew 500 ppm
Jun 20 07:10:42 dist-test-slave-wjw3 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939442, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:42.834757 27456 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:10:42.835552 27456 webserver.cc:460] Webserver started at http://127.8.24.62:40565/ using document root <none> and password file <none>
I20260620 07:10:42.836038 27456 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:42.836098 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:42.836266 27456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:42.838106 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "e4980356bf37447cab6c8449c35c565d"
format_stamp: "Formatted at 2026-06-20 07:10:42 on dist-test-slave-wjw3"
server_key: "0218cf633540143946126b5dba98167c"
server_key_iv: "bca30a42b0f3949bb4ee6ec0b5b3fe9b"
server_key_version: "encryptionkey@0"
I20260620 07:10:42.838614 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "e4980356bf37447cab6c8449c35c565d"
format_stamp: "Formatted at 2026-06-20 07:10:42 on dist-test-slave-wjw3"
server_key: "0218cf633540143946126b5dba98167c"
server_key_iv: "bca30a42b0f3949bb4ee6ec0b5b3fe9b"
server_key_version: "encryptionkey@0"
I20260620 07:10:42.841951 27456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260620 07:10:42.844125 27471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:42.845198 27456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 07:10:42.845299 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "e4980356bf37447cab6c8449c35c565d"
format_stamp: "Formatted at 2026-06-20 07:10:42 on dist-test-slave-wjw3"
server_key: "0218cf633540143946126b5dba98167c"
server_key_iv: "bca30a42b0f3949bb4ee6ec0b5b3fe9b"
server_key_version: "encryptionkey@0"
I20260620 07:10:42.845383 27456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:42.876178 27456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:42.878655 27456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:42.878813 27456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:42.885076 27456 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35419
I20260620 07:10:42.885092 27523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35419 every 8 connection(s)
I20260620 07:10:42.885965 27456 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:42.888507 27524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:42.890554 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27456
I20260620 07:10:42.890693 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:42.890952 8288 external_mini_cluster.cc:1428] Setting key 2832e5491f6a3e136c38417790b23c56
I20260620 07:10:42.894387 27524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: Bootstrap starting.
I20260620 07:10:42.896881 27524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:42.897679 27524 log.cc:824] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:42.899597 27524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: No bootstrap required, opened a new log
Jun 20 07:10:42 dist-test-slave-wjw3 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:42.903245 27524 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } }
I20260620 07:10:42.903520 27524 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:42.903581 27524 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4980356bf37447cab6c8449c35c565d, State: Initialized, Role: FOLLOWER
I20260620 07:10:42.904043 27524 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } }
I20260620 07:10:42.904192 27524 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:42.904258 27524 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:42.904354 27524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:42.905329 27524 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } }
I20260620 07:10:42.905658 27524 leader_election.cc:302] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4980356bf37447cab6c8449c35c565d; no voters:
I20260620 07:10:42.905881 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:42.892248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.8.24.62:35419)
0620 07:10:42.892657 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:42.892674 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:42.892715 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:10:42.893525 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:42.893561 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:42.893951 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:42.894293 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:42.894959 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:42.896430 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:42.897256 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:42.897932 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:42.898089 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:42.900542 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:42.900549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:42.900565 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:42.900594 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:42.902951 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:42.903316 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:42.903323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:42.903327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:42.903482 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:42.903784 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:42.903787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:42.903788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:42.904248 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:42.904443 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:42.904690 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 07:10:42.904967 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":107,"threads_started":1}
I20260620 07:10:42.905956 27524 leader_election.cc:288] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:42.906064 27529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:42.906271 27529 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [term 1 LEADER]: Becoming Leader. State: Replica: e4980356bf37447cab6c8449c35c565d, State: Running, Role: LEADER
I20260620 07:10:42.906584 27529 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } }
I20260620 07:10:42.907092 27524 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:42.908119 27530 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4980356bf37447cab6c8449c35c565d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } } }
I20260620 07:10:42.908239 27530 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:42.908592 27531 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4980356bf37447cab6c8449c35c565d. Latest consensus state: current_term: 1 leader_uuid: "e4980356bf37447cab6c8449c35c565d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4980356bf37447cab6c8449c35c565d" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35419 } } }
I20260620 07:10:42.908730 27531 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d [sys.catalog]: This master's current role is: LEADER
W20260620 07:10:42.911286 27544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:10:42.911347 27544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:10:42.911427 27538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:42.912103 27538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:42.916883 27538 catalog_manager.cc:1347] Generated new cluster ID: 20c64f9612fe4f62a5c92d683c53f696
I20260620 07:10:42.916944 27538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:42.926062 27538 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:42.927175 27538 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:42.935261 27538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: Generated new TSK 0
I20260620 07:10:42.935966 27538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.24.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
Entry for principal kudu/127.8.24.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab.
I20260620 07:10:42.963848 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:43.067906 27550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:43.068140 27550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:43.068194 27550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:43.072058 27550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:43.072108 27550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:43.072184 27550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:43.077178 27550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.1.keytab
--principal=kudu/127.8.24.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:43.078150 27550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:43.079101 27550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:43.087899 27556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.087908 27558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.087908 27555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:43.087956 27550 server_base.cc:1034] running on GCE node
I20260620 07:10:43.088469 27550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:43.089046 27550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:43.090197 27550 hybrid_clock.cc:648] HybridClock initialized: now 1781939443090182 us; error 27 us; skew 500 ppm
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:43.092694 27550 init.cc:377] Logged in from keytab as kudu/127.8.24.1@KRBTEST.COM (short username kudu)
I20260620 07:10:43.093500 27550 webserver.cc:460] Webserver started at http://127.8.24.1:46653/ using document root <none> and password file <none>
I20260620 07:10:43.094000 27550 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:43.094061 27550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:43.094245 27550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:43.096616 27550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "08dcd1e0df504454952ddb0b31924d31"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a2e471a7c024eed327924a589e8beca2"
server_key_iv: "cc46b356625d2f5447a08fb752bcbd61"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.097160 27550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "08dcd1e0df504454952ddb0b31924d31"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a2e471a7c024eed327924a589e8beca2"
server_key_iv: "cc46b356625d2f5447a08fb752bcbd61"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.101190 27550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260620 07:10:43.103273 27565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.104365 27550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:43.104461 27550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "08dcd1e0df504454952ddb0b31924d31"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a2e471a7c024eed327924a589e8beca2"
server_key_iv: "cc46b356625d2f5447a08fb752bcbd61"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.104560 27550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:43.122200 27550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:43.124693 27550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:43.124854 27550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:43.125378 27550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:43.126181 27550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:43.126227 27550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.126277 27550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:43.126307 27550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.134337 27550 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:41043
I20260620 07:10:43.134368 27678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:41043 every 8 connection(s)
I20260620 07:10:43.135142 27550 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:43.139218 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27550
I20260620 07:10:43.139345 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:43.139629 8288 external_mini_cluster.cc:1428] Setting key 88ce5b8dea0ec4f90db86072b4a1c688
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.1@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.24.2@KRBTEST.COM; defaulting to no policy
I20260620 07:10:43.149457 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.136720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:57311 (local address 127.8.24.62:35419)
0620 07:10:43.136833 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.136837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.137594 (+ 757us) server_negotiation.cc:407] Connection header received
0620 07:10:43.138806 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.138809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.138865 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.138939 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:43.140200 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.140904 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.142386 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.142682 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.142772 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.145392 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:43.145396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:43.145398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:43.145427 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:43.147168 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.147804 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.147808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.147810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.147894 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.148225 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.148228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.148230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.148421 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:43.148511 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.149214 (+ 703us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.149339 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:10:43.150743 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.137050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35419 (local address 127.8.24.1:57311)
0620 07:10:43.137450 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.137490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.138611 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.139112 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.139119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.139542 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:43.140068 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.140086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.141022 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.141026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.142277 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.142288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.142817 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.142820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.142952 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.143498 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:43.143518 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:43.145233 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:43.147295 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.147301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.147314 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.147692 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.147994 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.147997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.147999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.148127 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.148564 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:43.148570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:43.148860 (+ 290us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.149639 (+ 779us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.149965 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.2@KRBTEST.COM" created.
I20260620 07:10:43.151894 27679 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35419
I20260620 07:10:43.152107 27679 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:43.152547 27679 heartbeater.cc:511] Master 127.8.24.62:35419 requested a full tablet report, sending...
I20260620 07:10:43.153996 27488 ts_manager.cc:194] Registered new tserver with Master: 08dcd1e0df504454952ddb0b31924d31 (127.8.24.1:41043)
I20260620 07:10:43.155120 27488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.1@KRBTEST.COM'} at 127.8.24.1:57311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
Entry for principal kudu/127.8.24.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab.
I20260620 07:10:43.164350 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:43.270622 27684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:43.270843 27684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:43.270886 27684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:43.274777 27684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:43.274825 27684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:43.274893 27684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:43.279791 27684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.2.keytab
--principal=kudu/127.8.24.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:43.280659 27684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:43.281545 27684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:43.290015 27692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.290015 27690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.290005 27689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:43.290275 27684 server_base.cc:1034] running on GCE node
I20260620 07:10:43.290632 27684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:43.291162 27684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:43.292302 27684 hybrid_clock.cc:648] HybridClock initialized: now 1781939443292282 us; error 31 us; skew 500 ppm
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:43.294626 27684 init.cc:377] Logged in from keytab as kudu/127.8.24.2@KRBTEST.COM (short username kudu)
I20260620 07:10:43.295343 27684 webserver.cc:460] Webserver started at http://127.8.24.2:44783/ using document root <none> and password file <none>
I20260620 07:10:43.295851 27684 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:43.295913 27684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:43.296103 27684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:43.297907 27684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "7aefe97318dd4ab483c98cd6ede17b89"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "f0996f9e5d4bf130ecd2069f233b1a57"
server_key_iv: "f59ad92d157ac57c12baa4489be9432e"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.298390 27684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "7aefe97318dd4ab483c98cd6ede17b89"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "f0996f9e5d4bf130ecd2069f233b1a57"
server_key_iv: "f59ad92d157ac57c12baa4489be9432e"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.301672 27684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260620 07:10:43.303745 27699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.304780 27684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:43.304873 27684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "7aefe97318dd4ab483c98cd6ede17b89"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "f0996f9e5d4bf130ecd2069f233b1a57"
server_key_iv: "f59ad92d157ac57c12baa4489be9432e"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.304955 27684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:43.317307 27684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:43.319736 27684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:43.319878 27684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:43.320353 27684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:43.321041 27684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:43.321092 27684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.321141 27684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:43.321163 27684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.328823 27684 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:38165
I20260620 07:10:43.328846 27812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:38165 every 8 connection(s)
I20260620 07:10:43.329582 27684 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.2@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:43.339293 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27684
I20260620 07:10:43.339428 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:43.339738 8288 external_mini_cluster.cc:1428] Setting key dab345b47761db1ac6f82cb50911307d
I20260620 07:10:43.343127 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.331074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:41715 (local address 127.8.24.62:35419)
0620 07:10:43.331193 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.331196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.331917 (+ 721us) server_negotiation.cc:407] Connection header received
0620 07:10:43.332724 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.332727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.332781 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.332851 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:43.333891 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.334417 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.335485 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.335712 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.335801 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.338227 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:43.338231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:43.338233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:43.338273 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:43.340759 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.341345 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.341348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.341366 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.341465 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.341856 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.341859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.341861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.342104 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:43.342177 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.342828 (+ 651us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.342952 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:10:43.344312 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.331375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35419 (local address 127.8.24.2:41715)
0620 07:10:43.331797 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.331834 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.332542 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.333011 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.333018 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.333320 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:43.333761 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.333773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.334536 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.334538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.335354 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.335362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.335842 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.335846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.335992 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.336411 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:43.336425 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:43.338059 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:43.340872 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.340878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.340890 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.341227 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.341566 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.341569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.341570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.341741 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.342256 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:43.342262 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:43.342550 (+ 288us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.343291 (+ 741us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.343621 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260620 07:10:43.345554 27813 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35419
I20260620 07:10:43.345791 27813 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:43.346334 27813 heartbeater.cc:511] Master 127.8.24.62:35419 requested a full tablet report, sending...
I20260620 07:10:43.347479 27488 ts_manager.cc:194] Registered new tserver with Master: 7aefe97318dd4ab483c98cd6ede17b89 (127.8.24.2:38165)
I20260620 07:10:43.348229 27488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.2@KRBTEST.COM'} at 127.8.24.2:41715
WARNING: no policy specified for kudu/127.8.24.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
Entry for principal kudu/127.8.24.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab.
I20260620 07:10:43.363831 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:43.466904 27818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:43.467139 27818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:43.467177 27818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:43.471043 27818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:43.471097 27818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:43.471151 27818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:43.476461 27818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.3.keytab
--principal=kudu/127.8.24.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:35419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:43.477451 27818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:43.478385 27818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:10:43.487242 27818 server_base.cc:1034] running on GCE node
W20260620 07:10:43.487179 27823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.487200 27824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:43.487195 27826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:43.487761 27818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:43.488329 27818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:43.489470 27818 hybrid_clock.cc:648] HybridClock initialized: now 1781939443489440 us; error 47 us; skew 500 ppm
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:43.491983 27818 init.cc:377] Logged in from keytab as kudu/127.8.24.3@KRBTEST.COM (short username kudu)
I20260620 07:10:43.492769 27818 webserver.cc:460] Webserver started at http://127.8.24.3:35667/ using document root <none> and password file <none>
I20260620 07:10:43.493285 27818 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:43.493346 27818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:43.493619 27818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:43.495608 27818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "57cbeab293e64ceb808e8577a0adb073"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a67350a3138696214cdbfd6d41021975"
server_key_iv: "9c2fd52051e547a9c6dc872818d902b5"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.496124 27818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "57cbeab293e64ceb808e8577a0adb073"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a67350a3138696214cdbfd6d41021975"
server_key_iv: "9c2fd52051e547a9c6dc872818d902b5"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.499558 27818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:43.502019 27833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.503127 27818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:43.503226 27818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "57cbeab293e64ceb808e8577a0adb073"
format_stamp: "Formatted at 2026-06-20 07:10:43 on dist-test-slave-wjw3"
server_key: "a67350a3138696214cdbfd6d41021975"
server_key_iv: "9c2fd52051e547a9c6dc872818d902b5"
server_key_version: "encryptionkey@0"
I20260620 07:10:43.503314 27818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:43.513650 27818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:43.516500 27818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:43.516659 27818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:43.517180 27818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:43.517917 27818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:43.517962 27818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.518011 27818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:43.518046 27818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:43.525903 27818 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:41395
I20260620 07:10:43.525924 27946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:41395 every 8 connection(s)
I20260620 07:10:43.526728 27818 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:43.528311 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27818
I20260620 07:10:43.528436 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:43.528688 8288 external_mini_cluster.cc:1428] Setting key 8c597a8939acbc0b66f1d7476b28335f
Jun 20 07:10:43 dist-test-slave-wjw3 krb5kdc[27443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.3@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:10:43.538568 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.528355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54891 (local address 127.8.24.62:35419)
0620 07:10:43.528493 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.528497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.529274 (+ 777us) server_negotiation.cc:407] Connection header received
0620 07:10:43.530490 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.530493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.530546 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.530614 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:43.531656 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.532166 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.533194 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.533396 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.533459 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.535543 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:43.535547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:43.535550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:43.535575 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:43.536790 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.537230 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.537233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.537235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.537306 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.537598 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.537601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.537602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.537769 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:43.537851 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.538299 (+ 448us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.538413 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 07:10:43.539305 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.528704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35419 (local address 127.8.24.3:54891)
0620 07:10:43.529146 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.529196 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.530329 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.530752 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.530759 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.531078 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:43.531560 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.531574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.532246 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.532248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.533112 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.533120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.533490 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.533492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.533632 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.534079 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:10:43.534094 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:43.535379 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:43.536893 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.536898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.536911 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.537143 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.537381 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:10:43.537384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:10:43.537386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:10:43.537511 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:10:43.537861 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:43.537867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:43.538091 (+ 224us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.538245 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.538549 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
I20260620 07:10:43.540328 27947 heartbeater.cc:347] Connected to a master server at 127.8.24.62:35419
I20260620 07:10:43.540536 27947 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:43.541007 27947 heartbeater.cc:511] Master 127.8.24.62:35419 requested a full tablet report, sending...
I20260620 07:10:43.541996 27487 ts_manager.cc:194] Registered new tserver with Master: 57cbeab293e64ceb808e8577a0adb073 (127.8.24.3:41395)
I20260620 07:10:43.542568 27487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.24.3@KRBTEST.COM'} at 127.8.24.3:54891
I20260620 07:10:43.553601 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:43.561754 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.555342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.8.24.62:35419)
0620 07:10:43.555485 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.555490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.555644 (+ 154us) server_negotiation.cc:407] Connection header received
0620 07:10:43.555755 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.555758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.555799 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.555860 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:43.556492 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.556994 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.557786 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.557995 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.558085 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.559248 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:43.559252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:43.559254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:43.559279 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:43.560293 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.560656 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.560660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.560662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.560733 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:43.560957 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:43.560960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:43.560962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:43.561182 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:43.561300 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.561503 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.561610 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:43.564213 27487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:43.567034 27487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:43.580232 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.574626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38165 (local address 127.0.0.1:52376)
0620 07:10:43.575213 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.575226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.575316 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.576534 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.576537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.576554 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:43.576640 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.576646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577870 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.577872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.578881 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.578890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579877 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.579880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.579967 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.579985 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.580044 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.580094 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":36,"thread_start_us":96,"threads_started":1}
I20260620 07:10:43.580435 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.575299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.8.24.3:41395)
0620 07:10:43.575746 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.575756 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.575773 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:10:43.575828 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.575832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.575964 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.576288 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:43.576432 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577277 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.578785 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579598 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.580055 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.580174 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.580218 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.580295 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":97,"threads_started":1}
I20260620 07:10:43.580233 27961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.574874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:41395 (local address 127.0.0.1:41938)
0620 07:10:43.575404 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.575415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.575512 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.576156 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.576160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.576175 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:43.576339 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.576347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577384 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.577387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.578657 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.578667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579728 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.579732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.579831 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.579856 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.579926 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.580077 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":68,"thread_start_us":79,"threads_started":1}
I20260620 07:10:43.580581 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.574659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.8.24.1:41043)
0620 07:10:43.575632 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.575643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.575665 (+ 22us) server_negotiation.cc:407] Connection header received
0620 07:10:43.575738 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.575744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.575942 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.576109 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:43.576443 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577582 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.578974 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579547 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.579643 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.579786 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.580068 (+ 282us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.580159 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":117,"threads_started":1}
I20260620 07:10:43.580687 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.575676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.8.24.2:38165)
0620 07:10:43.576110 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.576120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.576140 (+ 20us) server_negotiation.cc:407] Connection header received
0620 07:10:43.576207 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.576212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.576414 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.576563 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:43.576726 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577758 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.578997 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579773 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.579859 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.579980 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.580434 (+ 454us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.580525 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":135,"threads_started":1}
I20260620 07:10:43.580991 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.574513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:41043 (local address 127.0.0.1:35854)
0620 07:10:43.575129 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.575176 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.575300 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.576091 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.576097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.576121 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:43.576311 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.576318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.577704 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.577707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.578856 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.578865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.579685 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.579689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.579801 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.579850 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.579945 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.580873 (+ 928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":57,"thread_start_us":60,"threads_started":1}
I20260620 07:10:43.583148 27881 tablet_service.cc:1467] Processing CreateTablet for tablet f1c65ce39be845d6b2941d2703306d98 (DEFAULT_TABLE table=test-table [id=a8d5e4eeb7c644e7bf9dc9a2a824e20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:43.583299 27613 tablet_service.cc:1467] Processing CreateTablet for tablet f1c65ce39be845d6b2941d2703306d98 (DEFAULT_TABLE table=test-table [id=a8d5e4eeb7c644e7bf9dc9a2a824e20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:43.583302 27747 tablet_service.cc:1467] Processing CreateTablet for tablet f1c65ce39be845d6b2941d2703306d98 (DEFAULT_TABLE table=test-table [id=a8d5e4eeb7c644e7bf9dc9a2a824e20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:43.584105 27881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c65ce39be845d6b2941d2703306d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:43.584285 27747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c65ce39be845d6b2941d2703306d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:43.584285 27613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c65ce39be845d6b2941d2703306d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:43.590107 27964 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Bootstrap starting.
I20260620 07:10:43.591087 27965 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Bootstrap starting.
I20260620 07:10:43.591463 27966 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Bootstrap starting.
I20260620 07:10:43.592356 27964 tablet_bootstrap.cc:654] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:43.593554 27964 log.cc:824] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:43.593879 27966 tablet_bootstrap.cc:654] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:43.594600 27966 log.cc:824] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:43.595391 27965 tablet_bootstrap.cc:654] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:43.595552 27964 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: No bootstrap required, opened a new log
I20260620 07:10:43.595775 27964 ts_tablet_manager.cc:1397] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:10:43.596457 27965 log.cc:824] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:43.596484 27966 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: No bootstrap required, opened a new log
I20260620 07:10:43.596650 27966 ts_tablet_manager.cc:1397] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 07:10:43.597843 27965 tablet_bootstrap.cc:492] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: No bootstrap required, opened a new log
I20260620 07:10:43.597993 27965 ts_tablet_manager.cc:1397] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260620 07:10:43.599480 27964 raft_consensus.cc:348] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.599747 27964 raft_consensus.cc:374] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:43.599828 27964 raft_consensus.cc:729] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57cbeab293e64ceb808e8577a0adb073, State: Initialized, Role: FOLLOWER
I20260620 07:10:43.600265 27964 consensus_queue.cc:260] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.600183 27966 raft_consensus.cc:348] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.600417 27966 raft_consensus.cc:374] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:43.600494 27966 raft_consensus.cc:729] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08dcd1e0df504454952ddb0b31924d31, State: Initialized, Role: FOLLOWER
I20260620 07:10:43.600885 27947 heartbeater.cc:503] Master 127.8.24.62:35419 was elected leader, sending a full tablet report...
I20260620 07:10:43.600739 27965 raft_consensus.cc:348] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.600970 27965 raft_consensus.cc:374] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:43.601011 27965 raft_consensus.cc:729] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aefe97318dd4ab483c98cd6ede17b89, State: Initialized, Role: FOLLOWER
I20260620 07:10:43.600993 27966 consensus_queue.cc:260] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.601224 27964 ts_tablet_manager.cc:1428] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:43.601392 27965 consensus_queue.cc:260] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.601716 27679 heartbeater.cc:503] Master 127.8.24.62:35419 was elected leader, sending a full tablet report...
I20260620 07:10:43.601838 27966 ts_tablet_manager.cc:1428] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 07:10:43.602223 27965 ts_tablet_manager.cc:1428] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260620 07:10:43.602285 27813 heartbeater.cc:503] Master 127.8.24.62:35419 was elected leader, sending a full tablet report...
W20260620 07:10:43.636220 27680 tablet.cc:2369] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:43.752931 27970 raft_consensus.cc:482] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:43.753139 27970 raft_consensus.cc:504] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.754329 27970 leader_election.cc:288] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08dcd1e0df504454952ddb0b31924d31 (127.8.24.1:41043), 7aefe97318dd4ab483c98cd6ede17b89 (127.8.24.2:38165)
I20260620 07:10:43.758946 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.754583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38165 (local address 127.8.24.3:33503)
0620 07:10:43.754776 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.754790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.754892 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.755169 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.755172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.755188 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:43.755304 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.755311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.756328 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.756331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.757678 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.757689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.758511 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.758515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.758631 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.758654 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.758734 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.758812 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":71}
I20260620 07:10:43.758993 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.754817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:33503 (local address 127.8.24.2:38165)
0620 07:10:43.754948 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.754952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.754965 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:10:43.755013 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.755015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.755060 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.755143 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:43.755413 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.756229 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.757796 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.758397 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.758499 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.758627 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.758750 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.758826 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:10:43.759181 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.754657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:37981 (local address 127.8.24.1:41043)
0620 07:10:43.754774 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.754779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.755067 (+ 288us) server_negotiation.cc:407] Connection header received
0620 07:10:43.755152 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.755154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.755199 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.755265 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:43.755507 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.756230 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.757796 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.758398 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.758499 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.758624 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.758987 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.759072 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 07:10:43.759552 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.754583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:41043 (local address 127.8.24.3:37981)
0620 07:10:43.754969 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.754981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:43.755057 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:43.755281 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:43.755284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:43.755297 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:43.755393 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.755399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.756324 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.756327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.757678 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:43.757689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.758723 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.758728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:43.758837 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.758860 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:10:43.759306 (+ 446us) client_negotiation.cc:240] Negotiation successful
0620 07:10:43.759381 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":100,"thread_start_us":122,"threads_started":1}
I20260620 07:10:43.759749 27767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c65ce39be845d6b2941d2703306d98" candidate_uuid: "57cbeab293e64ceb808e8577a0adb073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aefe97318dd4ab483c98cd6ede17b89" is_pre_election: true
I20260620 07:10:43.760078 27767 raft_consensus.cc:2454] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57cbeab293e64ceb808e8577a0adb073 in term 0.
I20260620 07:10:43.760106 27633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c65ce39be845d6b2941d2703306d98" candidate_uuid: "57cbeab293e64ceb808e8577a0adb073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08dcd1e0df504454952ddb0b31924d31" is_pre_election: true
I20260620 07:10:43.760381 27633 raft_consensus.cc:2454] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57cbeab293e64ceb808e8577a0adb073 in term 0.
I20260620 07:10:43.760578 27835 leader_election.cc:302] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57cbeab293e64ceb808e8577a0adb073, 7aefe97318dd4ab483c98cd6ede17b89; no voters:
I20260620 07:10:43.760857 27970 raft_consensus.cc:2781] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:43.760934 27970 raft_consensus.cc:482] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:43.760963 27970 raft_consensus.cc:3037] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:43.761927 27970 raft_consensus.cc:504] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.762271 27970 leader_election.cc:288] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [CANDIDATE]: Term 1 election: Requested vote from peers 08dcd1e0df504454952ddb0b31924d31 (127.8.24.1:41043), 7aefe97318dd4ab483c98cd6ede17b89 (127.8.24.2:38165)
I20260620 07:10:43.762578 27633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c65ce39be845d6b2941d2703306d98" candidate_uuid: "57cbeab293e64ceb808e8577a0adb073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08dcd1e0df504454952ddb0b31924d31"
I20260620 07:10:43.762633 27767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c65ce39be845d6b2941d2703306d98" candidate_uuid: "57cbeab293e64ceb808e8577a0adb073" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aefe97318dd4ab483c98cd6ede17b89"
I20260620 07:10:43.762699 27633 raft_consensus.cc:3037] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:43.762758 27767 raft_consensus.cc:3037] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:43.763670 27633 raft_consensus.cc:2454] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57cbeab293e64ceb808e8577a0adb073 in term 1.
I20260620 07:10:43.763753 27767 raft_consensus.cc:2454] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57cbeab293e64ceb808e8577a0adb073 in term 1.
I20260620 07:10:43.763943 27834 leader_election.cc:302] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08dcd1e0df504454952ddb0b31924d31, 57cbeab293e64ceb808e8577a0adb073; no voters:
I20260620 07:10:43.764096 27970 raft_consensus.cc:2781] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:43.764274 27970 raft_consensus.cc:686] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 1 LEADER]: Becoming Leader. State: Replica: 57cbeab293e64ceb808e8577a0adb073, State: Running, Role: LEADER
I20260620 07:10:43.764600 27970 consensus_queue.cc:237] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } }
I20260620 07:10:43.767530 27488 catalog_manager.cc:5495] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57cbeab293e64ceb808e8577a0adb073 (127.8.24.3). New cstate: current_term: 1 leader_uuid: "57cbeab293e64ceb808e8577a0adb073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57cbeab293e64ceb808e8577a0adb073" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 41395 } health_report { overall_health: HEALTHY } } }
W20260620 07:10:43.777571 27948 tablet.cc:2369] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:43.812485 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.808993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.8.24.3:41395)
0620 07:10:43.809116 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:43.809120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:43.809134 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:10:43.809216 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:43.809219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:43.809276 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:43.809366 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:43.809642 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.810409 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.811208 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:43.811408 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:43.811512 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:43.811753 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:43.812161 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:43.812249 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:43.812275 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 07:10:43.812325 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:10:43.817996 27633 raft_consensus.cc:1261] T f1c65ce39be845d6b2941d2703306d98 P 08dcd1e0df504454952ddb0b31924d31 [term 1 FOLLOWER]: Refusing update from remote peer 57cbeab293e64ceb808e8577a0adb073: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:43.817996 27767 raft_consensus.cc:1261] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 1 FOLLOWER]: Refusing update from remote peer 57cbeab293e64ceb808e8577a0adb073: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:43.818630 27970 consensus_queue.cc:1035] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aefe97318dd4ab483c98cd6ede17b89" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 38165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:43.818826 27974 consensus_queue.cc:1035] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08dcd1e0df504454952ddb0b31924d31" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 41043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:43.834287 27488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:43.834429 27488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:43.836581 27488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4980356bf37447cab6c8449c35c565d: Sending DeleteTablet for 3 replicas of tablet f1c65ce39be845d6b2941d2703306d98
I20260620 07:10:43.837234 27747 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c65ce39be845d6b2941d2703306d98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:52376
I20260620 07:10:43.837249 27613 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c65ce39be845d6b2941d2703306d98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:35854
I20260620 07:10:43.837494 27881 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c65ce39be845d6b2941d2703306d98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:43 UTC) from {username='kudu', principal='kudu/127.8.24.62@KRBTEST.COM'} at 127.0.0.1:41938
I20260620 07:10:43.837870 27989 tablet_replica.cc:330] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: stopping tablet replica
I20260620 07:10:43.838121 27989 raft_consensus.cc:2229] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:10:43.838207 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27550
I20260620 07:10:43.838503 27989 raft_consensus.cc:2258] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:43.839733 27989 ts_tablet_manager.cc:1905] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:43.842289 27989 ts_tablet_manager.cc:1918] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:43.842375 27989 log.cc:1196] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/f1c65ce39be845d6b2941d2703306d98
I20260620 07:10:43.842727 27989 ts_tablet_manager.cc:1939] T f1c65ce39be845d6b2941d2703306d98 P 57cbeab293e64ceb808e8577a0adb073: Deleting consensus metadata
I20260620 07:10:43.843633 27987 tablet_replica.cc:330] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: stopping tablet replica
I20260620 07:10:43.843750 27475 catalog_manager.cc:4841] TS 57cbeab293e64ceb808e8577a0adb073 (127.8.24.3:41395): tablet f1c65ce39be845d6b2941d2703306d98 (table test-table [id=a8d5e4eeb7c644e7bf9dc9a2a824e20d]) successfully deleted
I20260620 07:10:43.843972 27987 raft_consensus.cc:2229] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:43.844282 27987 raft_consensus.cc:2258] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 07:10:43.845441 27472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:41043 (error 108)
I20260620 07:10:43.845623 27987 ts_tablet_manager.cc:1905] T f1c65ce39be845d6b2941d2703306d98 P 7aefe97318dd4ab483c98cd6ede17b89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:43.846146 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27684
I20260620 07:10:43.846326 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.846039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:41043 (local address 127.0.0.1:35866)
0620 07:10:43.846187 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.846249 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:41043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260620 07:10:43.846462 27472 catalog_manager.cc:4584] TS 08dcd1e0df504454952ddb0b31924d31 (127.8.24.1:41043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1c65ce39be845d6b2941d2703306d98: Network error: Client connection negotiation failed: client connection to 127.8.24.1:41043: connect: Connection refused (error 111)
I20260620 07:10:43.852227 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27818
I20260620 07:10:43.852697 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:43.852430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:38165 (local address 127.0.0.1:52386)
0620 07:10:43.852579 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:43.852626 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.2:38165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260620 07:10:43.857602 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27456
2026-06-20T07:10:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2196 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T07:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:43Z Disabled control of system clock
I20260620 07:10:43.884351 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45191
--webserver_interface=127.8.24.62
--webserver_port=0
--builtin_ntp_servers=127.8.24.20:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:45191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:43.987847 27994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:43.988075 27994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:43.988109 27994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:43.991948 27994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:43.991992 27994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:43.992007 27994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:43.992030 27994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:43.992059 27994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:43.996631 27994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:45191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:45191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:43.997642 27994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:43.998977 27994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:44.006654 28002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.006711 27999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.006735 28000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:44.007056 27994 server_base.cc:1034] running on GCE node
I20260620 07:10:44.007582 27994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:44.008591 27994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:44.009738 27994 hybrid_clock.cc:648] HybridClock initialized: now 1781939444009723 us; error 26 us; skew 500 ppm
I20260620 07:10:44.011394 27994 webserver.cc:460] Webserver started at http://127.8.24.62:39151/ using document root <none> and password file <none>
I20260620 07:10:44.011950 27994 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:44.012008 27994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:44.012198 27994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:44.014112 27994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "ab9eee260f5a444c932ea2eca815ca77"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "b91fd865874a366a795323917ca2bc9b"
server_key_iv: "c93aec053efb4086261de73a7eefc61d"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.014632 27994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "ab9eee260f5a444c932ea2eca815ca77"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "b91fd865874a366a795323917ca2bc9b"
server_key_iv: "c93aec053efb4086261de73a7eefc61d"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.018096 27994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:10:44.020375 28008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.021569 27994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:44.021675 27994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ab9eee260f5a444c932ea2eca815ca77"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "b91fd865874a366a795323917ca2bc9b"
server_key_iv: "c93aec053efb4086261de73a7eefc61d"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.021777 27994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:44.046339 27994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:44.047082 27994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:44.047261 27994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:44.053910 27994 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:45191
I20260620 07:10:44.053932 28060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:45191 every 8 connection(s)
I20260620 07:10:44.054831 27994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:44.057754 28061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:44.059262 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 27994
I20260620 07:10:44.059377 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:44.059717 8288 external_mini_cluster.cc:1428] Setting key 9335f24fad601c40537909bb568896b1
I20260620 07:10:44.064090 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: Bootstrap starting.
I20260620 07:10:44.066195 28061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:44.067106 28061 log.cc:824] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:44.068447 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.061160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.8.24.62:45191)
0620 07:10:44.061770 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.061784 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.061817 (+ 33us) server_negotiation.cc:407] Connection header received
0620 07:10:44.062480 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.062498 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.062871 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.063171 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:44.063613 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.064567 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.065417 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.065874 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.066014 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.066743 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:44.066748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:44.066760 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:44.067138 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:44.067191 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:44.067212 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:44.067295 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.067483 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.067747 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":127,"threads_started":1}
I20260620 07:10:44.068940 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: No bootstrap required, opened a new log
I20260620 07:10:44.072597 28061 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } }
I20260620 07:10:44.072834 28061 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:44.072912 28061 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9eee260f5a444c932ea2eca815ca77, State: Initialized, Role: FOLLOWER
I20260620 07:10:44.073348 28061 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } }
I20260620 07:10:44.073503 28061 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:44.073577 28061 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:44.073670 28061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:44.074486 28061 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } }
I20260620 07:10:44.074802 28061 leader_election.cc:302] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab9eee260f5a444c932ea2eca815ca77; no voters:
I20260620 07:10:44.075066 28061 leader_election.cc:288] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:44.075167 28066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:44.075364 28066 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [term 1 LEADER]: Becoming Leader. State: Replica: ab9eee260f5a444c932ea2eca815ca77, State: Running, Role: LEADER
I20260620 07:10:44.075726 28066 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } }
I20260620 07:10:44.076177 28061 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:44.077101 28068 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9eee260f5a444c932ea2eca815ca77. Latest consensus state: current_term: 1 leader_uuid: "ab9eee260f5a444c932ea2eca815ca77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } } }
I20260620 07:10:44.077214 28068 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:44.077514 28067 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab9eee260f5a444c932ea2eca815ca77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9eee260f5a444c932ea2eca815ca77" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 45191 } } }
I20260620 07:10:44.077664 28067 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:44.077679 28075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:44.080304 28075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:44.085603 28075 catalog_manager.cc:1347] Generated new cluster ID: f4f85e4711d94ad89296a5d6b0ccf1e9
I20260620 07:10:44.085659 28075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:44.093351 28075 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:44.094451 28075 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:44.105394 28075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: Generated new TSK 0
I20260620 07:10:44.106170 28075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:44.118273 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.1:0
--local_ip_for_outbound_sockets=127.8.24.1
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--builtin_ntp_servers=127.8.24.20:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 07:10:44.222082 28085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:44.222311 28085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:44.222360 28085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:44.226279 28085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:44.226323 28085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:44.226393 28085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.1
I20260620 07:10:44.230964 28085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.24.1
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.1
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:44.231931 28085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:44.233210 28085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:44.241842 28090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.241850 28093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:44.241961 28085 server_base.cc:1034] running on GCE node
W20260620 07:10:44.241850 28091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:44.242470 28085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:44.243064 28085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:44.244233 28085 hybrid_clock.cc:648] HybridClock initialized: now 1781939444244211 us; error 35 us; skew 500 ppm
I20260620 07:10:44.245882 28085 webserver.cc:460] Webserver started at http://127.8.24.1:46517/ using document root <none> and password file <none>
I20260620 07:10:44.246372 28085 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:44.246424 28085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:44.246591 28085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:44.248512 28085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data/instance:
uuid: "067dd35337f14c34842de65d9fdf0be7"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "d79fdbca73cfc603304e5803352c7741"
server_key_iv: "4d7d4d8ac381d77ffa9a1d0ba935eb01"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.249004 28085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance:
uuid: "067dd35337f14c34842de65d9fdf0be7"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "d79fdbca73cfc603304e5803352c7741"
server_key_iv: "4d7d4d8ac381d77ffa9a1d0ba935eb01"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.252331 28085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:44.254806 28099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.255978 28085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:10:44.256076 28085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal
uuid: "067dd35337f14c34842de65d9fdf0be7"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "d79fdbca73cfc603304e5803352c7741"
server_key_iv: "4d7d4d8ac381d77ffa9a1d0ba935eb01"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.256167 28085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:44.279769 28085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:44.280453 28085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:44.280627 28085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:44.281191 28085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:44.282104 28085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:44.282150 28085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.282199 28085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:44.282228 28085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.290182 28085 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.1:44253
I20260620 07:10:44.290211 28212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.1:44253 every 8 connection(s)
I20260620 07:10:44.291020 28085 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/data/info.pb
I20260620 07:10:44.293856 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28085
I20260620 07:10:44.293977 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-0/wal/instance
I20260620 07:10:44.294240 8288 external_mini_cluster.cc:1428] Setting key fdb5f1e059e5ec291a6472291f065d6b
I20260620 07:10:44.296032 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.2:0
--local_ip_for_outbound_sockets=127.8.24.2
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--builtin_ntp_servers=127.8.24.20:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:44.299814 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.292688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.1:39143 (local address 127.8.24.62:45191)
0620 07:10:44.292823 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.292828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.293876 (+ 1048us) server_negotiation.cc:407] Connection header received
0620 07:10:44.295307 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.295310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.295364 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.295456 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:44.296327 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.296852 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.297947 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.298163 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.298248 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.299354 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:44.299358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:44.299362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:44.299372 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:44.299390 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:44.299398 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:44.299475 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.299611 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.299674 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:10:44.300596 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.293120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45191 (local address 127.8.24.1:39143)
0620 07:10:44.293734 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.293773 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.295119 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.295624 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.295630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.295715 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:44.296199 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.296216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.296967 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.296970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.297836 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.297844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.298296 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.298300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.298450 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.299084 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:44.299086 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:44.299137 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:44.299213 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:44.299475 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:44.299484 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:44.299489 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.299602 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.299861 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":106,"threads_started":1}
I20260620 07:10:44.301687 28213 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45191
I20260620 07:10:44.301926 28213 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:44.302536 28213 heartbeater.cc:511] Master 127.8.24.62:45191 requested a full tablet report, sending...
I20260620 07:10:44.304175 28025 ts_manager.cc:194] Registered new tserver with Master: 067dd35337f14c34842de65d9fdf0be7 (127.8.24.1:44253)
I20260620 07:10:44.305718 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.1:39143
W20260620 07:10:44.403321 28216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:44.403592 28216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:44.403631 28216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:44.407510 28216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:44.407557 28216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:44.407621 28216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.2
I20260620 07:10:44.412258 28216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.24.2
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.2
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:44.413273 28216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:44.414566 28216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:44.423527 28222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.423538 28221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.423516 28224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:44.423671 28216 server_base.cc:1034] running on GCE node
I20260620 07:10:44.424129 28216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:44.424711 28216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:44.425876 28216 hybrid_clock.cc:648] HybridClock initialized: now 1781939444425856 us; error 31 us; skew 500 ppm
I20260620 07:10:44.427601 28216 webserver.cc:460] Webserver started at http://127.8.24.2:43357/ using document root <none> and password file <none>
I20260620 07:10:44.428154 28216 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:44.428215 28216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:44.428376 28216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:44.430284 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data/instance:
uuid: "1ac02328cbec402a845d77c9fe672e8e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "fcc027f2b34d3074cb7fb7c0f9a21aa9"
server_key_iv: "b57ae974891fccf25d5e305b7cf4654f"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.430784 28216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance:
uuid: "1ac02328cbec402a845d77c9fe672e8e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "fcc027f2b34d3074cb7fb7c0f9a21aa9"
server_key_iv: "b57ae974891fccf25d5e305b7cf4654f"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.434283 28216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:10:44.436614 28230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.437785 28216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:44.437889 28216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal
uuid: "1ac02328cbec402a845d77c9fe672e8e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "fcc027f2b34d3074cb7fb7c0f9a21aa9"
server_key_iv: "b57ae974891fccf25d5e305b7cf4654f"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.437999 28216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:44.471136 28216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:44.471899 28216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:44.472083 28216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:44.472679 28216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:44.473635 28216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:44.473682 28216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.473733 28216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:44.473763 28216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.482093 28216 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.2:33429
I20260620 07:10:44.482125 28343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.2:33429 every 8 connection(s)
I20260620 07:10:44.482940 28216 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/data/info.pb
I20260620 07:10:44.491650 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.484655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.2:54323 (local address 127.8.24.62:45191)
0620 07:10:44.484786 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.484790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.485831 (+ 1041us) server_negotiation.cc:407] Connection header received
0620 07:10:44.487170 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.487173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.487228 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.487299 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:44.488236 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.488761 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.489883 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.490106 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.490195 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.491187 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:44.491190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:44.491192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:44.491204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:44.491223 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:44.491231 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:44.491287 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.491423 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.491505 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:10:44.491803 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28216
I20260620 07:10:44.491916 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal/instance
I20260620 07:10:44.492184 8288 external_mini_cluster.cc:1428] Setting key d6ea0dd899671a5ee1559dead3883083
I20260620 07:10:44.492549 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.485046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45191 (local address 127.8.24.2:54323)
0620 07:10:44.485690 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.485730 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.486970 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.487528 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.487535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.487623 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:44.488099 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.488111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.488883 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.488887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.489766 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.489776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.490251 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.490255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.490408 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.490943 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:44.490945 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:44.490986 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:44.491047 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:44.491298 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:44.491307 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:44.491313 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.491414 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.491726 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
I20260620 07:10:44.493610 28344 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45191
I20260620 07:10:44.493818 28344 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:44.494020 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.24.3:0
--local_ip_for_outbound_sockets=127.8.24.3
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--builtin_ntp_servers=127.8.24.20:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 07:10:44.494316 28344 heartbeater.cc:511] Master 127.8.24.62:45191 requested a full tablet report, sending...
I20260620 07:10:44.495235 28025 ts_manager.cc:194] Registered new tserver with Master: 1ac02328cbec402a845d77c9fe672e8e (127.8.24.2:33429)
I20260620 07:10:44.495798 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.2:54323
W20260620 07:10:44.603324 28347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:44.603616 28347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:44.603674 28347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:44.607836 28347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:44.607894 28347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:44.607966 28347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.24.3
I20260620 07:10:44.612694 28347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.24.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.24.3
--webserver_port=0
--tserver_master_addrs=127.8.24.62:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.24.3
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:44.613852 28347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:44.615206 28347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:44.624191 28355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.624185 28353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:44.624185 28352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:44.624404 28347 server_base.cc:1034] running on GCE node
I20260620 07:10:44.624894 28347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:44.625542 28347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:44.626731 28347 hybrid_clock.cc:648] HybridClock initialized: now 1781939444626707 us; error 36 us; skew 500 ppm
I20260620 07:10:44.628624 28347 webserver.cc:460] Webserver started at http://127.8.24.3:35529/ using document root <none> and password file <none>
I20260620 07:10:44.629187 28347 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:44.629252 28347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:44.629447 28347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:44.631491 28347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data/instance:
uuid: "abe14b1c97f84056ad9297cbbf40b15e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "35d82f9dae7cb042eba5145031de8530"
server_key_iv: "d9a9a1da2f71028d8bb92d3350261506"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.632012 28347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance:
uuid: "abe14b1c97f84056ad9297cbbf40b15e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "35d82f9dae7cb042eba5145031de8530"
server_key_iv: "d9a9a1da2f71028d8bb92d3350261506"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.635524 28347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:44.637817 28361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.638967 28347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:44.639068 28347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal
uuid: "abe14b1c97f84056ad9297cbbf40b15e"
format_stamp: "Formatted at 2026-06-20 07:10:44 on dist-test-slave-wjw3"
server_key: "35d82f9dae7cb042eba5145031de8530"
server_key_iv: "d9a9a1da2f71028d8bb92d3350261506"
server_key_version: "encryptionkey@0"
I20260620 07:10:44.639161 28347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:44.652608 28347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:44.653220 28347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:44.653369 28347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:44.653890 28347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 07:10:44.654748 28347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 07:10:44.654793 28347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.654843 28347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 07:10:44.654868 28347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:44.662653 28347 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.3:43639
I20260620 07:10:44.662708 28474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.3:43639 every 8 connection(s)
I20260620 07:10:44.663501 28347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/data/info.pb
I20260620 07:10:44.669574 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28347
I20260620 07:10:44.669684 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal/instance
I20260620 07:10:44.669958 8288 external_mini_cluster.cc:1428] Setting key 1ff205b784569a68c18f3e7a1bf4af1a
I20260620 07:10:44.672254 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.665231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:49695 (local address 127.8.24.62:45191)
0620 07:10:44.665374 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.665379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.666377 (+ 998us) server_negotiation.cc:407] Connection header received
0620 07:10:44.667651 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.667653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.667700 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.667767 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:44.668578 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.669106 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.670514 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.670742 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.670834 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.671835 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:44.671838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:44.671839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:44.671850 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:44.671869 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:44.671876 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:44.671930 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.672059 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.672118 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 07:10:44.673051 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.665633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:45191 (local address 127.8.24.3:49695)
0620 07:10:44.666236 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.666273 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.667467 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.667893 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.667899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.667978 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:10:44.668430 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.668441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.669204 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.669206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.670407 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.670417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.670881 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.670885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.671041 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.671602 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 07:10:44.671603 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:10:44.671641 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 07:10:44.671700 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:10:44.671940 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:10:44.671948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:10:44.671954 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.672054 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.672326 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260620 07:10:44.674055 28475 heartbeater.cc:347] Connected to a master server at 127.8.24.62:45191
I20260620 07:10:44.674263 28475 heartbeater.cc:464] Registering TS with master...
I20260620 07:10:44.674723 28475 heartbeater.cc:511] Master 127.8.24.62:45191 requested a full tablet report, sending...
I20260620 07:10:44.675617 28025 ts_manager.cc:194] Registered new tserver with Master: abe14b1c97f84056ad9297cbbf40b15e (127.8.24.3:43639)
I20260620 07:10:44.676143 28025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.24.3:49695
I20260620 07:10:44.683532 8288 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 07:10:44.688431 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.685301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.8.24.62:45191)
0620 07:10:44.685424 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.685429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.685581 (+ 152us) server_negotiation.cc:407] Connection header received
0620 07:10:44.685665 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.685667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.685705 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.685764 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:44.685936 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.686442 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.687195 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.687394 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.687483 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.688108 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:44.688111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:44.688113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:44.688123 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:44.688140 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:44.688147 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:44.688198 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.688252 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.688300 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:10:44.690750 28025 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 07:10:44.693671 28025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 07:10:44.706827 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.701336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.3:43639 (local address 127.0.0.1:59162)
0620 07:10:44.701663 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.701846 (+ 183us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.701971 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.702564 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.702570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.702596 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:44.702733 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.702740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.704028 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.704032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.705356 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.705366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706313 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.706423 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.706459 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.706566 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.706681 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260620 07:10:44.706835 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.701442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.8.24.3:43639)
0620 07:10:44.701920 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.701932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.701957 (+ 25us) server_negotiation.cc:407] Connection header received
0620 07:10:44.702178 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.702183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.702397 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.702549 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:44.702852 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.703929 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.705487 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706201 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.706481 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.706593 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.706686 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":168,"threads_started":1}
I20260620 07:10:44.707201 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.701648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44253 (local address 127.0.0.1:38258)
0620 07:10:44.702382 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.702393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.702470 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.703268 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.703271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.703285 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:44.703381 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.703387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.704589 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.704592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.705648 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.705658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706828 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.706917 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.706940 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.707003 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.707068 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":53,"thread_start_us":101,"threads_started":1}
I20260620 07:10:44.707535 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.702385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.8.24.2:33429)
0620 07:10:44.702773 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.702785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.703141 (+ 356us) server_negotiation.cc:407] Connection header received
0620 07:10:44.703219 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.703224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.703381 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.703599 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:44.703774 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.704571 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.705941 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706663 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706749 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.706890 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.707263 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.707360 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":87,"threads_started":1}
I20260620 07:10:44.707722 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.702087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.8.24.1:44253)
0620 07:10:44.702847 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.702858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.702881 (+ 23us) server_negotiation.cc:407] Connection header received
0620 07:10:44.702953 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.702958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.703153 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.703290 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:44.703482 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.704472 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.705759 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706478 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706589 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.706768 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.707480 (+ 712us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.707580 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":95,"threads_started":1}
I20260620 07:10:44.707748 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.702031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:33429 (local address 127.0.0.1:41430)
0620 07:10:44.703036 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.703047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.703132 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.703585 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.703588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.703605 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:44.703684 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.703690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.704664 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.704667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.705819 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.705828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.706940 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.706943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.707032 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.707051 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.707107 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.707163 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":29,"thread_start_us":74,"threads_started":1}
I20260620 07:10:44.710031 28278 tablet_service.cc:1467] Processing CreateTablet for tablet b4a4915caf9c443ab637709d299951a8 (DEFAULT_TABLE table=test-table [id=c233961e285b47858a63133164436206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:44.710064 28409 tablet_service.cc:1467] Processing CreateTablet for tablet b4a4915caf9c443ab637709d299951a8 (DEFAULT_TABLE table=test-table [id=c233961e285b47858a63133164436206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:44.710300 28147 tablet_service.cc:1467] Processing CreateTablet for tablet b4a4915caf9c443ab637709d299951a8 (DEFAULT_TABLE table=test-table [id=c233961e285b47858a63133164436206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 07:10:44.710978 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4a4915caf9c443ab637709d299951a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:44.711118 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4a4915caf9c443ab637709d299951a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:44.711118 28147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4a4915caf9c443ab637709d299951a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:44.717806 28493 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Bootstrap starting.
I20260620 07:10:44.717954 28492 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Bootstrap starting.
I20260620 07:10:44.718616 28494 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Bootstrap starting.
I20260620 07:10:44.720094 28493 tablet_bootstrap.cc:654] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:44.720094 28492 tablet_bootstrap.cc:654] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:44.720263 28494 tablet_bootstrap.cc:654] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:44.720889 28494 log.cc:824] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:44.720924 28492 log.cc:824] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:44.720923 28493 log.cc:824] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:44.722656 28494 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: No bootstrap required, opened a new log
I20260620 07:10:44.722671 28492 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: No bootstrap required, opened a new log
I20260620 07:10:44.722846 28494 ts_tablet_manager.cc:1397] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 07:10:44.722889 28492 ts_tablet_manager.cc:1397] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:44.723645 28493 tablet_bootstrap.cc:492] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: No bootstrap required, opened a new log
I20260620 07:10:44.723805 28493 ts_tablet_manager.cc:1397] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 07:10:44.726114 28492 raft_consensus.cc:348] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.726321 28492 raft_consensus.cc:374] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:44.726369 28492 raft_consensus.cc:729] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac02328cbec402a845d77c9fe672e8e, State: Initialized, Role: FOLLOWER
I20260620 07:10:44.726760 28492 consensus_queue.cc:260] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.726914 28494 raft_consensus.cc:348] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.727172 28494 raft_consensus.cc:374] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:44.727226 28494 raft_consensus.cc:729] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe14b1c97f84056ad9297cbbf40b15e, State: Initialized, Role: FOLLOWER
I20260620 07:10:44.727305 28493 raft_consensus.cc:348] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.727473 28344 heartbeater.cc:503] Master 127.8.24.62:45191 was elected leader, sending a full tablet report...
I20260620 07:10:44.727551 28493 raft_consensus.cc:374] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:44.727598 28493 raft_consensus.cc:729] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067dd35337f14c34842de65d9fdf0be7, State: Initialized, Role: FOLLOWER
I20260620 07:10:44.727725 28492 ts_tablet_manager.cc:1428] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:44.727684 28494 consensus_queue.cc:260] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.728088 28493 consensus_queue.cc:260] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.728600 28475 heartbeater.cc:503] Master 127.8.24.62:45191 was elected leader, sending a full tablet report...
I20260620 07:10:44.728874 28493 ts_tablet_manager.cc:1428] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 07:10:44.728871 28494 ts_tablet_manager.cc:1428] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 07:10:44.729148 28213 heartbeater.cc:503] Master 127.8.24.62:45191 was elected leader, sending a full tablet report...
W20260620 07:10:44.733780 28345 tablet.cc:2369] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:10:44.792166 28214 tablet.cc:2369] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:10:44.857072 28499 raft_consensus.cc:482] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 07:10:44.857259 28499 raft_consensus.cc:504] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.858320 28499 leader_election.cc:288] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ac02328cbec402a845d77c9fe672e8e (127.8.24.2:33429), 067dd35337f14c34842de65d9fdf0be7 (127.8.24.1:44253)
I20260620 07:10:44.862877 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.858608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:50031 (local address 127.8.24.2:33429)
0620 07:10:44.858732 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.858736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.858752 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:10:44.858856 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.858859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.858904 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.858980 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:44.859275 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.860108 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.861702 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.862271 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.862370 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.862429 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.862681 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.862747 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 07:10:44.862877 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.858717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.24.3:54879 (local address 127.8.24.1:44253)
0620 07:10:44.858830 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.858834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.859000 (+ 166us) server_negotiation.cc:407] Connection header received
0620 07:10:44.859140 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.859143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.859183 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.859250 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 07:10:44.859490 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.860181 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.861678 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.862270 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.862367 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.862429 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.862682 (+ 253us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.862747 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:10:44.862898 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.858505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.2:33429 (local address 127.8.24.3:50031)
0620 07:10:44.858660 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.858674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.858760 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.859010 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.859021 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.859042 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:44.859159 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.859165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.860202 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.860204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.861574 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.861585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.862384 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.862387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.862538 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.862597 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.862661 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.862745 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":97}
I20260620 07:10:44.862905 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.858549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44253 (local address 127.8.24.3:54879)
0620 07:10:44.858930 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.858940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 07:10:44.859053 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:10:44.859265 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:10:44.859268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:10:44.859283 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 07:10:44.859369 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.859373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.860263 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.860266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.861574 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:10:44.861585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.862382 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.862385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:10:44.862541 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.862590 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 07:10:44.862661 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 07:10:44.862745 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":96,"thread_start_us":101,"threads_started":1}
I20260620 07:10:44.863586 28298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4a4915caf9c443ab637709d299951a8" candidate_uuid: "abe14b1c97f84056ad9297cbbf40b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac02328cbec402a845d77c9fe672e8e" is_pre_election: true
I20260620 07:10:44.863623 28167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4a4915caf9c443ab637709d299951a8" candidate_uuid: "abe14b1c97f84056ad9297cbbf40b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067dd35337f14c34842de65d9fdf0be7" is_pre_election: true
I20260620 07:10:44.863862 28298 raft_consensus.cc:2454] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe14b1c97f84056ad9297cbbf40b15e in term 0.
I20260620 07:10:44.863869 28167 raft_consensus.cc:2454] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe14b1c97f84056ad9297cbbf40b15e in term 0.
I20260620 07:10:44.864290 28365 leader_election.cc:302] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067dd35337f14c34842de65d9fdf0be7, abe14b1c97f84056ad9297cbbf40b15e; no voters:
I20260620 07:10:44.864547 28499 raft_consensus.cc:2781] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 07:10:44.864638 28499 raft_consensus.cc:482] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 07:10:44.864681 28499 raft_consensus.cc:3037] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:44.865630 28499 raft_consensus.cc:504] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.865943 28499 leader_election.cc:288] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [CANDIDATE]: Term 1 election: Requested vote from peers 1ac02328cbec402a845d77c9fe672e8e (127.8.24.2:33429), 067dd35337f14c34842de65d9fdf0be7 (127.8.24.1:44253)
I20260620 07:10:44.866259 28298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4a4915caf9c443ab637709d299951a8" candidate_uuid: "abe14b1c97f84056ad9297cbbf40b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac02328cbec402a845d77c9fe672e8e"
I20260620 07:10:44.866259 28167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4a4915caf9c443ab637709d299951a8" candidate_uuid: "abe14b1c97f84056ad9297cbbf40b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067dd35337f14c34842de65d9fdf0be7"
I20260620 07:10:44.866374 28298 raft_consensus.cc:3037] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:44.866374 28167 raft_consensus.cc:3037] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:44.867571 28167 raft_consensus.cc:2454] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe14b1c97f84056ad9297cbbf40b15e in term 1.
I20260620 07:10:44.867565 28298 raft_consensus.cc:2454] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe14b1c97f84056ad9297cbbf40b15e in term 1.
I20260620 07:10:44.867827 28365 leader_election.cc:302] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067dd35337f14c34842de65d9fdf0be7, abe14b1c97f84056ad9297cbbf40b15e; no voters:
I20260620 07:10:44.867977 28499 raft_consensus.cc:2781] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:44.868171 28499 raft_consensus.cc:686] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 1 LEADER]: Becoming Leader. State: Replica: abe14b1c97f84056ad9297cbbf40b15e, State: Running, Role: LEADER
I20260620 07:10:44.868485 28499 consensus_queue.cc:237] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } }
I20260620 07:10:44.871306 28025 catalog_manager.cc:5495] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e reported cstate change: term changed from 0 to 1, leader changed from <none> to abe14b1c97f84056ad9297cbbf40b15e (127.8.24.3). New cstate: current_term: 1 leader_uuid: "abe14b1c97f84056ad9297cbbf40b15e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe14b1c97f84056ad9297cbbf40b15e" member_type: VOTER last_known_addr { host: "127.8.24.3" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 } health_report { overall_health: UNKNOWN } } }
I20260620 07:10:44.892598 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.889361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59176 (local address 127.8.24.3:43639)
0620 07:10:44.889484 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:44.889488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:44.889503 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:10:44.889585 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:44.889588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:44.889643 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:44.889728 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:10:44.889940 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.890440 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.891235 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:44.891471 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:44.891554 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:44.891831 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:10:44.892234 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:10:44.892336 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:44.892399 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 07:10:44.892446 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:10:44.897863 28298 raft_consensus.cc:1261] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 1 FOLLOWER]: Refusing update from remote peer abe14b1c97f84056ad9297cbbf40b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:44.897863 28167 raft_consensus.cc:1261] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 1 FOLLOWER]: Refusing update from remote peer abe14b1c97f84056ad9297cbbf40b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:10:44.898525 28502 consensus_queue.cc:1035] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "067dd35337f14c34842de65d9fdf0be7" member_type: VOTER last_known_addr { host: "127.8.24.1" port: 44253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:44.898684 28499 consensus_queue.cc:1035] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac02328cbec402a845d77c9fe672e8e" member_type: VOTER last_known_addr { host: "127.8.24.2" port: 33429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:10:44.905616 28507 mvcc.cc:205] Tried to move back new op lower bound from 7298823966295420928 to 7298823966183727104. Current Snapshot: MvccSnapshot[applied={T|T < 7298823966295420928}]
I20260620 07:10:44.907912 28506 mvcc.cc:205] Tried to move back new op lower bound from 7298823966295420928 to 7298823966183727104. Current Snapshot: MvccSnapshot[applied={T|T < 7298823966295420928}]
I20260620 07:10:44.912385 28023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:44.912509 28023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 07:10:44.914635 28023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab9eee260f5a444c932ea2eca815ca77: Sending DeleteTablet for 3 replicas of tablet b4a4915caf9c443ab637709d299951a8
I20260620 07:10:44.915316 28147 tablet_service.cc:1514] Processing DeleteTablet for tablet b4a4915caf9c443ab637709d299951a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:44 UTC) from {username='slave'} at 127.0.0.1:38258
I20260620 07:10:44.915355 28278 tablet_service.cc:1514] Processing DeleteTablet for tablet b4a4915caf9c443ab637709d299951a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:44 UTC) from {username='slave'} at 127.0.0.1:41430
I20260620 07:10:44.915611 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet b4a4915caf9c443ab637709d299951a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 07:10:44 UTC) from {username='slave'} at 127.0.0.1:59162
I20260620 07:10:44.915791 28516 tablet_replica.cc:330] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7: stopping tablet replica
I20260620 07:10:44.915827 28515 tablet_replica.cc:330] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: stopping tablet replica
I20260620 07:10:44.916075 28516 raft_consensus.cc:2229] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:44.916221 28515 raft_consensus.cc:2229] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 07:10:44.916314 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28085
I20260620 07:10:44.916366 28516 raft_consensus.cc:2258] T b4a4915caf9c443ab637709d299951a8 P 067dd35337f14c34842de65d9fdf0be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:44.916625 28515 raft_consensus.cc:2258] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:44.917235 28517 tablet_replica.cc:330] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: stopping tablet replica
I20260620 07:10:44.917510 28517 raft_consensus.cc:2229] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 1 LEADER]: Raft consensus shutting down.
I20260620 07:10:44.917857 28517 raft_consensus.cc:2258] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 07:10:44.918200 28515 ts_tablet_manager.cc:1905] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:44.919055 28517 ts_tablet_manager.cc:1905] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 07:10:44.920729 28515 ts_tablet_manager.cc:1918] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:44.920816 28515 log.cc:1196] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-1/wal/wals/b4a4915caf9c443ab637709d299951a8
I20260620 07:10:44.921087 28515 ts_tablet_manager.cc:1939] T b4a4915caf9c443ab637709d299951a8 P 1ac02328cbec402a845d77c9fe672e8e: Deleting consensus metadata
I20260620 07:10:44.921783 28517 ts_tablet_manager.cc:1918] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 07:10:44.921921 28517 log.cc:1196] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Deleting WAL directory at /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781939200335300-8288-0/minicluster-data/ts-2/wal/wals/b4a4915caf9c443ab637709d299951a8
I20260620 07:10:44.922118 28010 catalog_manager.cc:4841] TS 1ac02328cbec402a845d77c9fe672e8e (127.8.24.2:33429): tablet b4a4915caf9c443ab637709d299951a8 (table test-table [id=c233961e285b47858a63133164436206]) successfully deleted
I20260620 07:10:44.922508 28517 ts_tablet_manager.cc:1939] T b4a4915caf9c443ab637709d299951a8 P abe14b1c97f84056ad9297cbbf40b15e: Deleting consensus metadata
I20260620 07:10:44.923462 28010 catalog_manager.cc:4841] TS abe14b1c97f84056ad9297cbbf40b15e (127.8.24.3:43639): tablet b4a4915caf9c443ab637709d299951a8 (table test-table [id=c233961e285b47858a63133164436206]) successfully deleted
W20260620 07:10:44.924047 28012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.24.1:44253 (error 108)
I20260620 07:10:44.924585 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28216
I20260620 07:10:44.925232 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:44.924757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.1:44253 (local address 127.0.0.1:38270)
0620 07:10:44.925096 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0620 07:10:44.925154 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
W20260620 07:10:44.925491 28012 catalog_manager.cc:4584] TS 067dd35337f14c34842de65d9fdf0be7 (127.8.24.1:44253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4a4915caf9c443ab637709d299951a8: Network error: Client connection negotiation failed: client connection to 127.8.24.1:44253: connect: Connection refused (error 111)
I20260620 07:10:44.929545 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28347
I20260620 07:10:44.933956 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 27994
2026-06-20T07:10:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1075 ms)
[----------] 6 tests from EncryptionPolicyTest (9727 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:44 dist-test-slave-wjw3 krb5kdc[28521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:44 dist-test-slave-wjw3 krb5kdc[28521](info): set up 2 sockets
Jun 20 07:10:44 dist-test-slave-wjw3 krb5kdc[28521](info): commencing operation
krb5kdc: starting...
W20260620 07:10:45.895232 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:45 dist-test-slave-wjw3 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:46.001786 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:46.104773 28534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:46.105036 28534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:46.105083 28534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:46.109005 28534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:46.109061 28534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:46.109079 28534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:46.109097 28534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:46.109125 28534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:46.114213 28534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38877
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:46.115160 28534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:46.116127 28534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:46.123878 28539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:46.123968 28534 server_base.cc:1034] running on GCE node
W20260620 07:10:46.123915 28542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:46.123904 28540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:46.124580 28534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:46.125607 28534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:46.126751 28534 hybrid_clock.cc:648] HybridClock initialized: now 1781939446126727 us; error 37 us; skew 500 ppm
Jun 20 07:10:46 dist-test-slave-wjw3 krb5kdc[28521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939446, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:46.132972 28534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:46.133808 28534 webserver.cc:460] Webserver started at http://127.0.0.1:40003/ using document root <none> and password file <none>
I20260620 07:10:46.134316 28534 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:46.134379 28534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:46.134577 28534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:46.136585 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "bf7c0e37b2af4259886274547a83f7cc"
format_stamp: "Formatted at 2026-06-20 07:10:46 on dist-test-slave-wjw3"
server_key: "6a9ba6b86df694fe5929ab8f29ab9924"
server_key_iv: "ffc0153f471fc80ad0ce2d4ee1769a5a"
server_key_version: "encryptionkey@0"
I20260620 07:10:46.137110 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "bf7c0e37b2af4259886274547a83f7cc"
format_stamp: "Formatted at 2026-06-20 07:10:46 on dist-test-slave-wjw3"
server_key: "6a9ba6b86df694fe5929ab8f29ab9924"
server_key_iv: "ffc0153f471fc80ad0ce2d4ee1769a5a"
server_key_version: "encryptionkey@0"
I20260620 07:10:46.140585 28534 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:46.142753 28549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:46.143919 28534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:46.144037 28534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "bf7c0e37b2af4259886274547a83f7cc"
format_stamp: "Formatted at 2026-06-20 07:10:46 on dist-test-slave-wjw3"
server_key: "6a9ba6b86df694fe5929ab8f29ab9924"
server_key_iv: "ffc0153f471fc80ad0ce2d4ee1769a5a"
server_key_version: "encryptionkey@0"
I20260620 07:10:46.144126 28534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:46.166568 28534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:46.169615 28534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:46.169813 28534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:46.176748 28534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38877
I20260620 07:10:46.176757 28601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38877 every 8 connection(s)
I20260620 07:10:46.177677 28534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:46.180303 28602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:46.185992 28602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc: Bootstrap starting.
I20260620 07:10:46.186321 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28534
I20260620 07:10:46.186442 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:46.186691 8288 external_mini_cluster.cc:1428] Setting key 40b18c9247dcbed4730381a50381b30e
I20260620 07:10:46.188555 28602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:46.189431 28602 log.cc:824] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:46.191495 28602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc: No bootstrap required, opened a new log
I20260620 07:10:46.195290 28602 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260620 07:10:46.195575 28602 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:46.195633 28602 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7c0e37b2af4259886274547a83f7cc, State: Initialized, Role: FOLLOWER
Jun 20 07:10:46 dist-test-slave-wjw3 krb5kdc[28521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 07:10:46.196075 28602 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260620 07:10:46.196216 28602 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:46.196280 28602 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:46.196380 28602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:46.197405 28602 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260620 07:10:46.197768 28602 leader_election.cc:302] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf7c0e37b2af4259886274547a83f7cc; no voters:
I20260620 07:10:46.198092 28602 leader_election.cc:288] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:46.198182 28607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:46.198370 28607 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [term 1 LEADER]: Becoming Leader. State: Replica: bf7c0e37b2af4259886274547a83f7cc, State: Running, Role: LEADER
I20260620 07:10:46.198674 28607 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } }
I20260620 07:10:46.199666 28602 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:46.200084 28609 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf7c0e37b2af4259886274547a83f7cc. Latest consensus state: current_term: 1 leader_uuid: "bf7c0e37b2af4259886274547a83f7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } } }
I20260620 07:10:46.200222 28609 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:46.200613 28608 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf7c0e37b2af4259886274547a83f7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7c0e37b2af4259886274547a83f7cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38877 } } }
I20260620 07:10:46.200736 28608 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:46.201025 28611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:46.202917 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:46.188002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38625 (local address 127.0.0.1:38877)
0620 07:10:46.188408 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:46.188432 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:46.188474 (+ 42us) server_negotiation.cc:407] Connection header received
0620 07:10:46.189297 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:46.189322 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:46.189761 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:46.190055 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:46.190747 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:46.192186 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:46.193000 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:46.193693 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:46.193843 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:46.196436 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:46.196442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:46.196460 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:46.196492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:46.198529 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:46.199298 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:46.199303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:46.199309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:46.199564 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:46.200061 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:46.200064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:46.200066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:46.200636 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:46.201572 (+ 936us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:46.201762 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 07:10:46.201995 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":104,"threads_started":1}
I20260620 07:10:46.204553 28611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:46.206192 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:46.210204 28611 catalog_manager.cc:1347] Generated new cluster ID: 56676c38ff7c403e9f190ccb6249c105
I20260620 07:10:46.210265 28611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:46.215368 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:46.207995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59041 (local address 127.0.0.1:38877)
0620 07:10:46.208173 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:46.208178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:46.208213 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:10:46.208332 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:46.208335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:46.208383 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:46.208442 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:46.209071 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:46.210167 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:46.211009 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:46.211251 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:46.211334 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:46.212567 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:46.212572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:46.212573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:46.212593 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:46.213821 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:46.214242 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:46.214245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:46.214246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:46.214311 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:46.214593 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:46.214596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:46.214598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:46.214747 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:46.214884 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:46.215142 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 07:10:46.215219 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260620 07:10:46.248514 28611 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:46.249423 28611 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:46.262027 28611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf7c0e37b2af4259886274547a83f7cc: Generated new TSK 0
I20260620 07:10:46.262689 28611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:46.272708 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28534
W20260620 07:10:46.277364 28519 connection.cc:535] client connection to 127.0.0.1:38877 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38877): Connection reset by peer (error 104)
2026-06-20T07:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:46 dist-test-slave-wjw3 krb5kdc[28633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:46 dist-test-slave-wjw3 krb5kdc[28633](info): set up 2 sockets
Jun 20 07:10:46 dist-test-slave-wjw3 krb5kdc[28633](info): commencing operation
krb5kdc: starting...
W20260620 07:10:47.227270 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:47 dist-test-slave-wjw3 krb5kdc[28633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:47.334185 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:47.438648 28646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:47.438890 28646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:47.438946 28646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:47.442883 28646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:47.442951 28646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:47.442978 28646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:47.442999 28646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:47.443023 28646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:47.448243 28646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:47.449451 28646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:47.450433 28646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:47.458451 28651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:47.458455 28652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:47.458582 28654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:47.458645 28646 server_base.cc:1034] running on GCE node
I20260620 07:10:47.459180 28646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:47.460275 28646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:47.461433 28646 hybrid_clock.cc:648] HybridClock initialized: now 1781939447461418 us; error 26 us; skew 500 ppm
Jun 20 07:10:47 dist-test-slave-wjw3 krb5kdc[28633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939447, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:47.463950 28646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:47.464735 28646 webserver.cc:460] Webserver started at http://127.0.0.1:43023/ using document root <none> and password file <none>
I20260620 07:10:47.465229 28646 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:47.465283 28646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:47.465441 28646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:47.467327 28646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "707e8b25b0c241f5a8288fc8d6419407"
format_stamp: "Formatted at 2026-06-20 07:10:47 on dist-test-slave-wjw3"
server_key: "ee5855546914fce312fa4266f4de1be5"
server_key_iv: "79abccdebb1013bdab9f9145e9f9308d"
server_key_version: "encryptionkey@0"
I20260620 07:10:47.467864 28646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "707e8b25b0c241f5a8288fc8d6419407"
format_stamp: "Formatted at 2026-06-20 07:10:47 on dist-test-slave-wjw3"
server_key: "ee5855546914fce312fa4266f4de1be5"
server_key_iv: "79abccdebb1013bdab9f9145e9f9308d"
server_key_version: "encryptionkey@0"
I20260620 07:10:47.471333 28646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:47.473663 28661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:47.474860 28646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:47.474961 28646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "707e8b25b0c241f5a8288fc8d6419407"
format_stamp: "Formatted at 2026-06-20 07:10:47 on dist-test-slave-wjw3"
server_key: "ee5855546914fce312fa4266f4de1be5"
server_key_iv: "79abccdebb1013bdab9f9145e9f9308d"
server_key_version: "encryptionkey@0"
I20260620 07:10:47.475070 28646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:47.493893 28646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:47.496583 28646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:47.496743 28646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:47.503005 28646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42405
I20260620 07:10:47.503054 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42405 every 8 connection(s)
I20260620 07:10:47.503935 28646 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:47.506491 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:47.509603 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28646
I20260620 07:10:47.509719 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:47.509964 8288 external_mini_cluster.cc:1428] Setting key c4727f7e433ed6c938d0684cdef431cf
I20260620 07:10:47.512173 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407: Bootstrap starting.
I20260620 07:10:47.514818 28714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:47.515655 28714 log.cc:824] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:47.517686 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407: No bootstrap required, opened a new log
I20260620 07:10:47.519223 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:47.511238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33201 (local address 127.0.0.1:42405)
0620 07:10:47.511650 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:47.511668 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:47.511710 (+ 42us) server_negotiation.cc:407] Connection header received
0620 07:10:47.512546 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:47.512568 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:47.512999 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:47.513358 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:47.513792 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:47.515214 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:47.516124 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:47.516663 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:47.516780 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:47.517613 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:47.517618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:47.517630 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:47.518030 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:47.518082 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:47.518105 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:47.518198 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:47.518373 (+ 175us) server_negotiation.cc:299] Negotiation successful
0620 07:10:47.518501 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":118,"threads_started":1}
I20260620 07:10:47.521433 28714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260620 07:10:47.521688 28714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:47.521746 28714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707e8b25b0c241f5a8288fc8d6419407, State: Initialized, Role: FOLLOWER
I20260620 07:10:47.522063 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:47.522197 28714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260620 07:10:47.522339 28714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:47.522393 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:47.522480 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:47.523607 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260620 07:10:47.523952 28714 leader_election.cc:302] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707e8b25b0c241f5a8288fc8d6419407; no voters:
I20260620 07:10:47.524274 28714 leader_election.cc:288] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:47.524339 28723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:47.524523 28723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [term 1 LEADER]: Becoming Leader. State: Replica: 707e8b25b0c241f5a8288fc8d6419407, State: Running, Role: LEADER
I20260620 07:10:47.524818 28723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } }
I20260620 07:10:47.525496 28714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:47.526430 28726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707e8b25b0c241f5a8288fc8d6419407. Latest consensus state: current_term: 1 leader_uuid: "707e8b25b0c241f5a8288fc8d6419407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } } }
I20260620 07:10:47.526541 28726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:47.526773 28725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707e8b25b0c241f5a8288fc8d6419407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e8b25b0c241f5a8288fc8d6419407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42405 } } }
I20260620 07:10:47.526837 28725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:47.526901 28733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:47.528920 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:47.524222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60635 (local address 127.0.0.1:42405)
0620 07:10:47.524329 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:47.524333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:47.524694 (+ 361us) server_negotiation.cc:407] Connection header received
0620 07:10:47.524974 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:47.524977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:47.525019 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:47.525079 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:47.525332 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:47.525962 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:47.527149 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:47.527477 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:47.527584 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:47.528409 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:47.528413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:47.528415 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:47.528427 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:47.528451 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:47.528463 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:47.528557 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:47.528762 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 07:10:47.528798 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:10:47.529898 28733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:47.534909 28733 catalog_manager.cc:1347] Generated new cluster ID: c80cd3531ef946c4810c87a400f0549a
I20260620 07:10:47.534965 28733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:47.542464 28733 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:47.543345 28733 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:47.548252 28733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707e8b25b0c241f5a8288fc8d6419407: Generated new TSK 0
I20260620 07:10:47.548949 28733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:47.556463 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28646
W20260620 07:10:47.561103 28631 connection.cc:535] client connection to 127.0.0.1:42405 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42405): Connection reset by peer (error 104)
2026-06-20T07:10:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:47 dist-test-slave-wjw3 krb5kdc[28745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:47 dist-test-slave-wjw3 krb5kdc[28745](info): set up 2 sockets
Jun 20 07:10:47 dist-test-slave-wjw3 krb5kdc[28745](info): commencing operation
krb5kdc: starting...
W20260620 07:10:48.499325 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:48 dist-test-slave-wjw3 krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:48.608484 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:48.714423 28758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:48.714679 28758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:48.714737 28758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:48.718658 28758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:48.718715 28758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:48.718741 28758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:48.718763 28758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:48.718791 28758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:48.723860 28758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:48.724838 28758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:48.725803 28758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:48.733588 28764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:48.733690 28758 server_base.cc:1034] running on GCE node
W20260620 07:10:48.733567 28766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:48.733578 28763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:48.734368 28758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:48.735375 28758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:48.736541 28758 hybrid_clock.cc:648] HybridClock initialized: now 1781939448736518 us; error 36 us; skew 500 ppm
Jun 20 07:10:48 dist-test-slave-wjw3 krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939448, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:48.739048 28758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:48.739889 28758 webserver.cc:460] Webserver started at http://127.0.0.1:39583/ using document root <none> and password file <none>
I20260620 07:10:48.740404 28758 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:48.740468 28758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:48.740654 28758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:48.742604 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "8c36f50e561d43f89252fd1ff34ea171"
format_stamp: "Formatted at 2026-06-20 07:10:48 on dist-test-slave-wjw3"
server_key: "93b854ac38b7ea794ba73b70ce25feff"
server_key_iv: "41d6583b0acac665960b65ae09911118"
server_key_version: "encryptionkey@0"
I20260620 07:10:48.743129 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "8c36f50e561d43f89252fd1ff34ea171"
format_stamp: "Formatted at 2026-06-20 07:10:48 on dist-test-slave-wjw3"
server_key: "93b854ac38b7ea794ba73b70ce25feff"
server_key_iv: "41d6583b0acac665960b65ae09911118"
server_key_version: "encryptionkey@0"
I20260620 07:10:48.746630 28758 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 07:10:48.748955 28773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:48.750066 28758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:48.750172 28758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "8c36f50e561d43f89252fd1ff34ea171"
format_stamp: "Formatted at 2026-06-20 07:10:48 on dist-test-slave-wjw3"
server_key: "93b854ac38b7ea794ba73b70ce25feff"
server_key_iv: "41d6583b0acac665960b65ae09911118"
server_key_version: "encryptionkey@0"
I20260620 07:10:48.750262 28758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:48.754527 28758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:48.757210 28758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:48.757359 28758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:48.763549 28758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43659
I20260620 07:10:48.763569 28825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43659 every 8 connection(s)
I20260620 07:10:48.764380 28758 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:48.766952 28826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:48.772167 28826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171: Bootstrap starting.
I20260620 07:10:48.773584 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28758
I20260620 07:10:48.773725 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:48.774013 8288 external_mini_cluster.cc:1428] Setting key b9927e86129dc053618d115ae40fd4d5
I20260620 07:10:48.774724 28826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:48.775532 28826 log.cc:824] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:48.777370 28826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171: No bootstrap required, opened a new log
I20260620 07:10:48.778632 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:48.775021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42365 (local address 127.0.0.1:43659)
0620 07:10:48.775390 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:48.775404 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:48.775459 (+ 55us) server_negotiation.cc:407] Connection header received
0620 07:10:48.776154 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:48.776172 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:48.776435 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:48.776704 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:48.777501 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:48.777513 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:48.777524 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:48.777535 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:48.777586 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:48.777605 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:48.777682 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:48.777815 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 07:10:48.777958 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":93,"threads_started":1}
I20260620 07:10:48.781304 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:48.781131 28826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } }
I20260620 07:10:48.781373 28826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:48.781433 28826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c36f50e561d43f89252fd1ff34ea171, State: Initialized, Role: FOLLOWER
I20260620 07:10:48.781863 28826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } }
I20260620 07:10:48.782012 28826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:48.782074 28826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:48.782169 28826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:48.783206 28826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } }
I20260620 07:10:48.783576 28826 leader_election.cc:302] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c36f50e561d43f89252fd1ff34ea171; no voters:
I20260620 07:10:48.783890 28826 leader_election.cc:288] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:48.783957 28836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:48.784132 28836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [term 1 LEADER]: Becoming Leader. State: Replica: 8c36f50e561d43f89252fd1ff34ea171, State: Running, Role: LEADER
I20260620 07:10:48.784402 28836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } }
I20260620 07:10:48.784982 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:48.783370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45789 (local address 127.0.0.1:43659)
0620 07:10:48.783500 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:48.783512 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:48.783680 (+ 168us) server_negotiation.cc:407] Connection header received
0620 07:10:48.783817 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:48.783818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:48.783847 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:48.783912 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:48.784604 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:48.784608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:48.784610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:48.784622 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:48.784649 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:48.784662 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:48.784711 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:48.784857 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 07:10:48.784892 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 07:10:48.785099 28826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:48.786458 28838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c36f50e561d43f89252fd1ff34ea171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } } }
I20260620 07:10:48.786592 28838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:48.786952 28848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:48.786911 28837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c36f50e561d43f89252fd1ff34ea171. Latest consensus state: current_term: 1 leader_uuid: "8c36f50e561d43f89252fd1ff34ea171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c36f50e561d43f89252fd1ff34ea171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43659 } } }
I20260620 07:10:48.787019 28837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:48.790055 28848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:48.795104 28848 catalog_manager.cc:1347] Generated new cluster ID: 0d47c241d9c944e1b09cb4b5fb609fe4
I20260620 07:10:48.795166 28848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:48.867477 28848 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:48.867851 28848 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:48.876993 28848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c36f50e561d43f89252fd1ff34ea171: Generated new TSK 0
I20260620 07:10:48.877648 28848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:48.894448 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28758
2026-06-20T07:10:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:48 dist-test-slave-wjw3 krb5kdc[28857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:48 dist-test-slave-wjw3 krb5kdc[28857](info): set up 2 sockets
Jun 20 07:10:48 dist-test-slave-wjw3 krb5kdc[28857](info): commencing operation
krb5kdc: starting...
W20260620 07:10:49.835354 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:49 dist-test-slave-wjw3 krb5kdc[28857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:49.943856 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:50.047137 28870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:50.047396 28870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:50.047475 28870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:50.051357 28870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:50.051409 28870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:50.051452 28870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:50.051476 28870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:50.051499 28870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:50.056528 28870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:50.057478 28870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:50.058398 28870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:50.066246 28875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:50.066237 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:50.066234 28876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:50.066419 28870 server_base.cc:1034] running on GCE node
I20260620 07:10:50.067003 28870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:50.068018 28870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:50.069168 28870 hybrid_clock.cc:648] HybridClock initialized: now 1781939450069141 us; error 38 us; skew 500 ppm
Jun 20 07:10:50 dist-test-slave-wjw3 krb5kdc[28857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:50.071635 28870 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:50.072429 28870 webserver.cc:460] Webserver started at http://127.0.0.1:35669/ using document root <none> and password file <none>
I20260620 07:10:50.072952 28870 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:50.073011 28870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:50.073203 28870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:50.075201 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "5c2cfb06eb81403db09eb497fe51f3ca"
format_stamp: "Formatted at 2026-06-20 07:10:50 on dist-test-slave-wjw3"
server_key: "c9c365b189f4b9ced6cd27adeb4e660e"
server_key_iv: "48acfca6d1ba1200075c76628784053b"
server_key_version: "encryptionkey@0"
I20260620 07:10:50.075721 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5c2cfb06eb81403db09eb497fe51f3ca"
format_stamp: "Formatted at 2026-06-20 07:10:50 on dist-test-slave-wjw3"
server_key: "c9c365b189f4b9ced6cd27adeb4e660e"
server_key_iv: "48acfca6d1ba1200075c76628784053b"
server_key_version: "encryptionkey@0"
I20260620 07:10:50.079010 28870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:50.081197 28885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:50.082273 28870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:50.082408 28870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "5c2cfb06eb81403db09eb497fe51f3ca"
format_stamp: "Formatted at 2026-06-20 07:10:50 on dist-test-slave-wjw3"
server_key: "c9c365b189f4b9ced6cd27adeb4e660e"
server_key_iv: "48acfca6d1ba1200075c76628784053b"
server_key_version: "encryptionkey@0"
I20260620 07:10:50.082495 28870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:50.106279 28870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:50.109334 28870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:50.109506 28870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:50.115731 28870 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38585
I20260620 07:10:50.115778 28937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38585 every 8 connection(s)
I20260620 07:10:50.116611 28870 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:50.119040 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28870
I20260620 07:10:50.119153 28938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:50.119164 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:50.119454 8288 external_mini_cluster.cc:1428] Setting key e3e94f9ba3de93e4fce70d87c1644c24
I20260620 07:10:50.125463 28938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca: Bootstrap starting.
Jun 20 07:10:50 dist-test-slave-wjw3 krb5kdc[28857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 07:10:50.127918 28938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:50.128583 28938 log.cc:824] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:50.130616 28938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca: No bootstrap required, opened a new log
I20260620 07:10:50.133098 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:50.120651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38825 (local address 127.0.0.1:38585)
0620 07:10:50.121025 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:50.121038 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:50.121072 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:10:50.121663 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:50.121680 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:50.121999 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:50.122253 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:50.123086 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:50.124114 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:50.125259 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:50.125739 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:50.125855 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:50.128355 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:50.128360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:50.128373 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:50.128399 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:50.130153 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:50.130576 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:50.130582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:50.130587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:50.130724 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:50.131003 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:50.131007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:50.131008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:50.131496 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:50.131682 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:50.131872 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 07:10:50.132148 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":117,"threads_started":1}
I20260620 07:10:50.134842 28938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } }
I20260620 07:10:50.135097 28938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:50.135181 28938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2cfb06eb81403db09eb497fe51f3ca, State: Initialized, Role: FOLLOWER
I20260620 07:10:50.135612 28938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } }
I20260620 07:10:50.135732 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:50.135746 28938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:50.135803 28938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:50.135892 28938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:50.136682 28938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } }
I20260620 07:10:50.137075 28938 leader_election.cc:302] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c2cfb06eb81403db09eb497fe51f3ca; no voters:
I20260620 07:10:50.137359 28938 leader_election.cc:288] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:50.137451 28947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:50.137671 28947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [term 1 LEADER]: Becoming Leader. State: Replica: 5c2cfb06eb81403db09eb497fe51f3ca, State: Running, Role: LEADER
I20260620 07:10:50.138123 28947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } }
I20260620 07:10:50.138578 28938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:50.139348 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c2cfb06eb81403db09eb497fe51f3ca. Latest consensus state: current_term: 1 leader_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } } }
I20260620 07:10:50.139542 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:50.139991 28957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:50.140153 28950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c2cfb06eb81403db09eb497fe51f3ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38585 } } }
I20260620 07:10:50.140318 28950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:50.143146 28957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:50.145819 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:50.137778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54101 (local address 127.0.0.1:38585)
0620 07:10:50.137865 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:50.137869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:50.137881 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:10:50.137913 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:50.137915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:50.137955 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:50.138032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:50.138795 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:50.139625 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:50.140695 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:50.141016 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:50.141119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:50.142588 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:50.142592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:50.142594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:50.142618 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:50.144134 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:50.144627 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:50.144630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:50.144632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:50.144716 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:50.145000 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:50.145003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:50.145005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:50.145203 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:50.145335 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:50.145564 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 07:10:50.145671 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 07:10:50.148716 28957 catalog_manager.cc:1347] Generated new cluster ID: 98b90fd0c97b491cbf1f7a93077b30b7
I20260620 07:10:50.148773 28957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:50.171402 28957 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:50.172638 28957 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:50.188447 28957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c2cfb06eb81403db09eb497fe51f3ca: Generated new TSK 0
I20260620 07:10:50.189247 28957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:50.193423 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28870
2026-06-20T07:10:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:50 dist-test-slave-wjw3 krb5kdc[28969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:50 dist-test-slave-wjw3 krb5kdc[28969](info): set up 2 sockets
Jun 20 07:10:50 dist-test-slave-wjw3 krb5kdc[28969](info): commencing operation
krb5kdc: starting...
W20260620 07:10:51.143332 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:51 dist-test-slave-wjw3 krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:51.254050 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:51.357698 28982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:51.357945 28982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:51.357991 28982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:51.361876 28982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:51.361927 28982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:51.361944 28982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:51.361964 28982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:51.361977 28982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:51.366968 28982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:51.367906 28982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:51.368886 28982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:51.376477 28988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:51.376468 28990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:51.376468 28987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:51.376736 28982 server_base.cc:1034] running on GCE node
I20260620 07:10:51.377362 28982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:51.378365 28982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:51.379503 28982 hybrid_clock.cc:648] HybridClock initialized: now 1781939451379471 us; error 40 us; skew 500 ppm
Jun 20 07:10:51 dist-test-slave-wjw3 krb5kdc[28969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:51.384711 28982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:51.385525 28982 webserver.cc:460] Webserver started at http://127.0.0.1:37685/ using document root <none> and password file <none>
I20260620 07:10:51.386034 28982 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:51.386098 28982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:51.386286 28982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:51.388273 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "5e3c5f5f82714ccfa55ce40fab00db40"
format_stamp: "Formatted at 2026-06-20 07:10:51 on dist-test-slave-wjw3"
server_key: "80ceed2a9a3f1e5fb01536d24abd9fce"
server_key_iv: "de39636f16ac6bfda3ee67d5a9c34c25"
server_key_version: "encryptionkey@0"
I20260620 07:10:51.388789 28982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "5e3c5f5f82714ccfa55ce40fab00db40"
format_stamp: "Formatted at 2026-06-20 07:10:51 on dist-test-slave-wjw3"
server_key: "80ceed2a9a3f1e5fb01536d24abd9fce"
server_key_iv: "de39636f16ac6bfda3ee67d5a9c34c25"
server_key_version: "encryptionkey@0"
I20260620 07:10:51.392222 28982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:51.394380 28997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:51.395509 28982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:10:51.395617 28982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "5e3c5f5f82714ccfa55ce40fab00db40"
format_stamp: "Formatted at 2026-06-20 07:10:51 on dist-test-slave-wjw3"
server_key: "80ceed2a9a3f1e5fb01536d24abd9fce"
server_key_iv: "de39636f16ac6bfda3ee67d5a9c34c25"
server_key_version: "encryptionkey@0"
I20260620 07:10:51.395701 28982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:51.418952 28982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:51.421872 28982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:51.422035 28982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:51.428251 28982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44227
I20260620 07:10:51.428263 29049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44227 every 8 connection(s)
I20260620 07:10:51.429111 28982 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:51.429359 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 28982
I20260620 07:10:51.429471 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:51.429760 8288 external_mini_cluster.cc:1428] Setting key aae4c700b01534759a3f1cf86097b5e4
I20260620 07:10:51.432142 29050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:51.438555 29050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40: Bootstrap starting.
I20260620 07:10:51.439404 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:51.431037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42321 (local address 127.0.0.1:44227)
0620 07:10:51.431428 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:51.431470 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:51.431511 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:10:51.432312 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:51.432334 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:51.432727 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:51.433091 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:51.433455 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:51.434891 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:51.435810 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:51.436488 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:51.436640 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:51.437423 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:51.437429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:51.437445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:51.437906 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:51.437972 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:51.438012 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:51.438104 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:51.438209 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 07:10:51.438541 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260620 07:10:51.440828 29050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:51.441617 29050 log.cc:824] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:51.441944 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:51.443414 29050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40: No bootstrap required, opened a new log
I20260620 07:10:51.447510 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:51.443850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51541 (local address 127.0.0.1:44227)
0620 07:10:51.443957 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:51.443960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:51.444155 (+ 195us) server_negotiation.cc:407] Connection header received
0620 07:10:51.444233 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:51.444237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:51.444279 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:51.444356 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:51.444557 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:51.445291 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:51.446030 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:51.446362 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:51.446455 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:51.447096 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:51.447099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:51.447102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:51.447113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:51.447138 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:51.447149 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:51.447220 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:51.447286 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 07:10:51.447334 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:10:51.447366 29050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260620 07:10:51.447665 29050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:51.447724 29050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3c5f5f82714ccfa55ce40fab00db40, State: Initialized, Role: FOLLOWER
I20260620 07:10:51.448163 29050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260620 07:10:51.448313 29050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:51.448376 29050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:51.448467 29050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:51.449257 29050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260620 07:10:51.449563 29050 leader_election.cc:302] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e3c5f5f82714ccfa55ce40fab00db40; no voters:
I20260620 07:10:51.449822 29050 leader_election.cc:288] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:51.449935 29060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:51.450136 29060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [term 1 LEADER]: Becoming Leader. State: Replica: 5e3c5f5f82714ccfa55ce40fab00db40, State: Running, Role: LEADER
I20260620 07:10:51.450445 29060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260620 07:10:51.450879 29050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:51.451867 29062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e3c5f5f82714ccfa55ce40fab00db40. Latest consensus state: current_term: 1 leader_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } } }
I20260620 07:10:51.452184 29062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:51.452091 29061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e3c5f5f82714ccfa55ce40fab00db40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } } }
I20260620 07:10:51.452477 29061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:51.452913 29070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:51.455776 29070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:51.460574 29070 catalog_manager.cc:1347] Generated new cluster ID: f3a93bebe2de484d8ca50571e23c7b91
I20260620 07:10:51.460634 29070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:51.488659 29070 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:51.489935 29070 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:51.499069 29070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e3c5f5f82714ccfa55ce40fab00db40: Generated new TSK 0
I20260620 07:10:51.499897 29070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:51.503979 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 28982
2026-06-20T07:10:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:51 dist-test-slave-wjw3 krb5kdc[29081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:51 dist-test-slave-wjw3 krb5kdc[29081](info): set up 2 sockets
Jun 20 07:10:51 dist-test-slave-wjw3 krb5kdc[29081](info): commencing operation
krb5kdc: starting...
W20260620 07:10:52.455415 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:52 dist-test-slave-wjw3 krb5kdc[29081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:52.563493 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:52.669282 29094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:52.669528 29094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:52.669570 29094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:52.673484 29094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:52.673542 29094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:52.673563 29094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:52.673579 29094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:52.673593 29094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:52.678793 29094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:52.679736 29094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:52.680663 29094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:52.688467 29099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:52.688449 29102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:52.688639 29094 server_base.cc:1034] running on GCE node
W20260620 07:10:52.688498 29100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:52.689240 29094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:52.690253 29094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:52.691407 29094 hybrid_clock.cc:648] HybridClock initialized: now 1781939452691388 us; error 34 us; skew 500 ppm
Jun 20 07:10:52 dist-test-slave-wjw3 krb5kdc[29081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939452, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:52.693938 29094 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:52.694743 29094 webserver.cc:460] Webserver started at http://127.0.0.1:43597/ using document root <none> and password file <none>
I20260620 07:10:52.695216 29094 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:52.695268 29094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:52.695417 29094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:52.697361 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "fb5016accb184241b535adf31c93dc3f"
format_stamp: "Formatted at 2026-06-20 07:10:52 on dist-test-slave-wjw3"
server_key: "325cfcab07d17c69a46d8a954f873d95"
server_key_iv: "4f7756a5a2252c8f5ff18956ee7ac384"
server_key_version: "encryptionkey@0"
I20260620 07:10:52.697865 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "fb5016accb184241b535adf31c93dc3f"
format_stamp: "Formatted at 2026-06-20 07:10:52 on dist-test-slave-wjw3"
server_key: "325cfcab07d17c69a46d8a954f873d95"
server_key_iv: "4f7756a5a2252c8f5ff18956ee7ac384"
server_key_version: "encryptionkey@0"
I20260620 07:10:52.701260 29094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:10:52.703495 29109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:52.704638 29094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:52.704731 29094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "fb5016accb184241b535adf31c93dc3f"
format_stamp: "Formatted at 2026-06-20 07:10:52 on dist-test-slave-wjw3"
server_key: "325cfcab07d17c69a46d8a954f873d95"
server_key_iv: "4f7756a5a2252c8f5ff18956ee7ac384"
server_key_version: "encryptionkey@0"
I20260620 07:10:52.704809 29094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:52.709859 29094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:52.712463 29094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:52.712603 29094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:52.718801 29094 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33851
I20260620 07:10:52.718814 29161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33851 every 8 connection(s)
I20260620 07:10:52.719668 29094 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:52.722170 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:52.727183 29162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f: Bootstrap starting.
I20260620 07:10:52.728297 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29094
I20260620 07:10:52.728413 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:52.728637 8288 external_mini_cluster.cc:1428] Setting key 1876d6812dfb56438e47a0bf65ad17bf
I20260620 07:10:52.729583 29162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:52.730314 29162 log.cc:824] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:52.732203 29162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f: No bootstrap required, opened a new log
I20260620 07:10:52.733335 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:52.729607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35947 (local address 127.0.0.1:33851)
0620 07:10:52.730019 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:52.730033 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:52.730068 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:10:52.730746 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:52.730769 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:52.731098 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:52.731399 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:52.732197 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:52.732201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:52.732213 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:52.732224 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:52.732286 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:52.732313 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:52.732403 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:52.732519 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 07:10:52.732677 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260620 07:10:52.735684 29162 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } }
I20260620 07:10:52.735910 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:52.735920 29162 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:52.735980 29162 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5016accb184241b535adf31c93dc3f, State: Initialized, Role: FOLLOWER
I20260620 07:10:52.736371 29162 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } }
I20260620 07:10:52.736490 29162 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:52.736548 29162 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:52.736634 29162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:52.737566 29162 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } }
I20260620 07:10:52.737920 29162 leader_election.cc:302] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb5016accb184241b535adf31c93dc3f; no voters:
I20260620 07:10:52.738237 29162 leader_election.cc:288] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:52.738309 29172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:52.738497 29172 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [term 1 LEADER]: Becoming Leader. State: Replica: fb5016accb184241b535adf31c93dc3f, State: Running, Role: LEADER
I20260620 07:10:52.739126 29172 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } }
I20260620 07:10:52.739580 29162 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:52.740912 29174 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb5016accb184241b535adf31c93dc3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } } }
I20260620 07:10:52.740912 29173 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb5016accb184241b535adf31c93dc3f. Latest consensus state: current_term: 1 leader_uuid: "fb5016accb184241b535adf31c93dc3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5016accb184241b535adf31c93dc3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33851 } } }
I20260620 07:10:52.741058 29174 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:52.741067 29173 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:52.741446 29183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:52.741667 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:52.737849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46463 (local address 127.0.0.1:33851)
0620 07:10:52.737948 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:52.737953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:52.738182 (+ 229us) server_negotiation.cc:407] Connection header received
0620 07:10:52.738287 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:52.738290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:52.738323 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:52.738387 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:52.739148 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:52.739152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:52.739155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:52.739166 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:52.739193 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:52.739204 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:52.739269 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:52.739428 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 07:10:52.739513 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 07:10:52.744882 29183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:52.750433 29183 catalog_manager.cc:1347] Generated new cluster ID: be773a98768447838c09d0df73183fcb
I20260620 07:10:52.750504 29183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:52.760012 29183 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:52.760336 29183 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:52.765262 29183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb5016accb184241b535adf31c93dc3f: Generated new TSK 0
I20260620 07:10:52.765874 29183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:52.771790 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29094
2026-06-20T07:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:52 dist-test-slave-wjw3 krb5kdc[29193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:52 dist-test-slave-wjw3 krb5kdc[29193](info): set up 2 sockets
Jun 20 07:10:52 dist-test-slave-wjw3 krb5kdc[29193](info): commencing operation
krb5kdc: starting...
W20260620 07:10:53.727327 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:53 dist-test-slave-wjw3 krb5kdc[29193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:53.835886 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:53.938263 29206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:53.938555 29206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:53.938598 29206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:53.942548 29206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:53.942605 29206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:53.942622 29206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:53.942636 29206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:53.942657 29206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:53.947888 29206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:53.948846 29206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:53.949813 29206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:53.957731 29212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:53.957743 29211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:53.957743 29214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:53.957793 29206 server_base.cc:1034] running on GCE node
I20260620 07:10:53.958549 29206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:53.959647 29206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:53.960813 29206 hybrid_clock.cc:648] HybridClock initialized: now 1781939453960785 us; error 48 us; skew 500 ppm
Jun 20 07:10:53 dist-test-slave-wjw3 krb5kdc[29193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:53.963387 29206 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:53.964258 29206 webserver.cc:460] Webserver started at http://127.0.0.1:36337/ using document root <none> and password file <none>
I20260620 07:10:53.964748 29206 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:53.964802 29206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:53.964964 29206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:53.966887 29206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "3d7ab1260e1f480aacd10565f1bbf772"
format_stamp: "Formatted at 2026-06-20 07:10:53 on dist-test-slave-wjw3"
server_key: "8b2f5320aa8829e46bb2930daf2a679b"
server_key_iv: "ed34fac3a9cb36ff75758712e9580543"
server_key_version: "encryptionkey@0"
I20260620 07:10:53.967387 29206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "3d7ab1260e1f480aacd10565f1bbf772"
format_stamp: "Formatted at 2026-06-20 07:10:53 on dist-test-slave-wjw3"
server_key: "8b2f5320aa8829e46bb2930daf2a679b"
server_key_iv: "ed34fac3a9cb36ff75758712e9580543"
server_key_version: "encryptionkey@0"
I20260620 07:10:53.970847 29206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:10:53.973160 29221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:53.974287 29206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:10:53.974380 29206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "3d7ab1260e1f480aacd10565f1bbf772"
format_stamp: "Formatted at 2026-06-20 07:10:53 on dist-test-slave-wjw3"
server_key: "8b2f5320aa8829e46bb2930daf2a679b"
server_key_iv: "ed34fac3a9cb36ff75758712e9580543"
server_key_version: "encryptionkey@0"
I20260620 07:10:53.974462 29206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:53.991291 29206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:53.994426 29206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:53.994577 29206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:54.001154 29206 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43077
I20260620 07:10:54.001173 29273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43077 every 8 connection(s)
I20260620 07:10:54.002040 29206 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:54.004671 29274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:54.009924 29274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: Bootstrap starting.
I20260620 07:10:54.010730 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29206
I20260620 07:10:54.010862 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:54.011147 8288 external_mini_cluster.cc:1428] Setting key a105790a80a203ce4198b92785004db1
I20260620 07:10:54.012660 29274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:54.013345 29274 log.cc:824] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:54.015367 29274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: No bootstrap required, opened a new log
I20260620 07:10:54.019562 29274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } }
I20260620 07:10:54.019785 29274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Jun 20 07:10:54 dist-test-slave-wjw3 krb5kdc[29193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 07:10:54.019845 29274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7ab1260e1f480aacd10565f1bbf772, State: Initialized, Role: FOLLOWER
I20260620 07:10:54.020282 29274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } }
I20260620 07:10:54.020437 29274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:54.020504 29274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:54.020619 29274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:54.021714 29274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } }
I20260620 07:10:54.022092 29274 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7ab1260e1f480aacd10565f1bbf772; no voters:
I20260620 07:10:54.022424 29274 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:54.022509 29279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:54.022709 29279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7ab1260e1f480aacd10565f1bbf772, State: Running, Role: LEADER
I20260620 07:10:54.023075 29279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } }
I20260620 07:10:54.023732 29274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:54.024740 29280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7ab1260e1f480aacd10565f1bbf772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } } }
I20260620 07:10:54.025105 29280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:54.025002 29281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7ab1260e1f480aacd10565f1bbf772. Latest consensus state: current_term: 1 leader_uuid: "3d7ab1260e1f480aacd10565f1bbf772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7ab1260e1f480aacd10565f1bbf772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43077 } } }
I20260620 07:10:54.025410 29281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772 [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:54.026115 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:54.012233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:50255 (local address 127.0.0.1:43077)
0620 07:10:54.012621 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:54.012635 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:54.012669 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:10:54.013321 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:54.013337 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:54.013711 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:54.014072 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:54.014876 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:54.015911 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:54.017079 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:54.017547 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:54.017661 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:54.020408 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:54.020414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:54.020429 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:54.020463 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:54.022385 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:54.022957 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:54.022962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:54.022968 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:54.023101 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:54.023473 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:54.023484 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:54.023486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:54.023977 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:54.024214 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:54.024911 (+ 697us) server_negotiation.cc:299] Negotiation successful
0620 07:10:54.025144 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
W20260620 07:10:54.028821 29294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:10:54.028893 29294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:10:54.028960 29289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:54.029002 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:54.029773 29289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:54.035386 29289 catalog_manager.cc:1347] Generated new cluster ID: c98941467b634723b16045847b94535b
I20260620 07:10:54.035486 29289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:54.040210 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:54.031849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:39009 (local address 127.0.0.1:43077)
0620 07:10:54.031937 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:54.031942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:54.032136 (+ 194us) server_negotiation.cc:407] Connection header received
0620 07:10:54.032240 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:54.032242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:54.032287 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:54.032364 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:54.033129 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:54.034091 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:54.034860 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:54.035114 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:54.035205 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:54.036861 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:54.036865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:54.036868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:54.036892 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:54.038413 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:54.038894 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:54.038897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:54.038899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:54.038981 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:54.039248 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:54.039251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:54.039253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:54.039474 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:54.039612 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:54.039974 (+ 362us) server_negotiation.cc:299] Negotiation successful
0620 07:10:54.040069 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 07:10:54.048902 29289 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:54.050078 29289 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:54.057489 29289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7ab1260e1f480aacd10565f1bbf772: Generated new TSK 0
I20260620 07:10:54.058310 29289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:54.066519 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29206
W20260620 07:10:54.071072 29191 connection.cc:535] client connection to 127.0.0.1:43077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43077): Connection reset by peer (error 104)
2026-06-20T07:10:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:54 dist-test-slave-wjw3 krb5kdc[29305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:54 dist-test-slave-wjw3 krb5kdc[29305](info): set up 2 sockets
Jun 20 07:10:54 dist-test-slave-wjw3 krb5kdc[29305](info): commencing operation
krb5kdc: starting...
W20260620 07:10:55.011346 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:55 dist-test-slave-wjw3 krb5kdc[29305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:55.120417 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:55.223816 29318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:55.224056 29318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:55.224094 29318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:55.227922 29318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:55.227965 29318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:55.227983 29318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:55.227998 29318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:55.228015 29318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:55.233119 29318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:55.234011 29318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:55.234905 29318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:55.242436 29324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:55.242496 29323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:55.242619 29318 server_base.cc:1034] running on GCE node
W20260620 07:10:55.242496 29326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:55.243209 29318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:55.244247 29318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:55.245431 29318 hybrid_clock.cc:648] HybridClock initialized: now 1781939455245410 us; error 30 us; skew 500 ppm
Jun 20 07:10:55 dist-test-slave-wjw3 krb5kdc[29305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939455, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:55.247936 29318 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:55.248780 29318 webserver.cc:460] Webserver started at http://127.0.0.1:45563/ using document root <none> and password file <none>
I20260620 07:10:55.249280 29318 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:55.249351 29318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:55.249542 29318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:55.251474 29318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "720f91f2d563468a86377848d68c3d3a"
format_stamp: "Formatted at 2026-06-20 07:10:55 on dist-test-slave-wjw3"
server_key: "519361bcd637c550cd0c627a860072a0"
server_key_iv: "43c3c4fe379d89fdb08a3301b387bde9"
server_key_version: "encryptionkey@0"
I20260620 07:10:55.251995 29318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "720f91f2d563468a86377848d68c3d3a"
format_stamp: "Formatted at 2026-06-20 07:10:55 on dist-test-slave-wjw3"
server_key: "519361bcd637c550cd0c627a860072a0"
server_key_iv: "43c3c4fe379d89fdb08a3301b387bde9"
server_key_version: "encryptionkey@0"
I20260620 07:10:55.255386 29318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:55.257589 29333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:55.258697 29318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 07:10:55.258806 29318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "720f91f2d563468a86377848d68c3d3a"
format_stamp: "Formatted at 2026-06-20 07:10:55 on dist-test-slave-wjw3"
server_key: "519361bcd637c550cd0c627a860072a0"
server_key_iv: "43c3c4fe379d89fdb08a3301b387bde9"
server_key_version: "encryptionkey@0"
I20260620 07:10:55.258908 29318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:55.273519 29318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:55.276010 29318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:55.276175 29318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:55.282500 29318 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37651
I20260620 07:10:55.282527 29385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37651 every 8 connection(s)
I20260620 07:10:55.283380 29318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:55.285180 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29318
I20260620 07:10:55.285295 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:55.285553 8288 external_mini_cluster.cc:1428] Setting key 7bb94b96fc1def7ae7264850ac2a588a
I20260620 07:10:55.286144 29386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:55.292224 29386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a: Bootstrap starting.
I20260620 07:10:55.294140 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:55.286832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:35981 (local address 127.0.0.1:37651)
0620 07:10:55.287250 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:55.287265 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:55.287300 (+ 35us) server_negotiation.cc:407] Connection header received
0620 07:10:55.287979 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:55.287995 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:55.288319 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:55.288590 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:55.288879 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:55.289820 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:55.291025 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:55.291533 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:55.291643 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:55.292498 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:55.292502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:55.292514 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:55.292921 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:55.292975 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:55.292997 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:55.293092 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:55.293276 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 07:10:55.293399 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":120,"threads_started":1}
I20260620 07:10:55.294409 29386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:55.295066 29386 log.cc:824] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:55.296738 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:55.296854 29386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a: No bootstrap required, opened a new log
I20260620 07:10:55.300956 29386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260620 07:10:55.301215 29386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:55.301270 29386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720f91f2d563468a86377848d68c3d3a, State: Initialized, Role: FOLLOWER
I20260620 07:10:55.301679 29386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260620 07:10:55.301808 29386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:55.301891 29386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:55.302001 29386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:55.302517 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:55.298742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:35349 (local address 127.0.0.1:37651)
0620 07:10:55.298856 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:55.298860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:55.299000 (+ 140us) server_negotiation.cc:407] Connection header received
0620 07:10:55.299081 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:55.299084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:55.299128 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:55.299223 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:55.299411 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:55.300176 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:55.300942 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:55.301257 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:55.301361 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:55.302094 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:55.302098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:55.302101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:55.302112 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:55.302138 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:55.302150 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:55.302215 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:55.302368 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 07:10:55.302397 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:10:55.302989 29386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260620 07:10:55.303294 29386 leader_election.cc:302] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 720f91f2d563468a86377848d68c3d3a; no voters:
I20260620 07:10:55.303575 29386 leader_election.cc:288] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:55.303654 29396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:55.303849 29396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [term 1 LEADER]: Becoming Leader. State: Replica: 720f91f2d563468a86377848d68c3d3a, State: Running, Role: LEADER
I20260620 07:10:55.304186 29396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260620 07:10:55.304646 29386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:55.305639 29398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 720f91f2d563468a86377848d68c3d3a. Latest consensus state: current_term: 1 leader_uuid: "720f91f2d563468a86377848d68c3d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } } }
I20260620 07:10:55.305729 29398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:55.305668 29397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "720f91f2d563468a86377848d68c3d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720f91f2d563468a86377848d68c3d3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } } }
I20260620 07:10:55.305769 29397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:55.306039 29405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:55.309262 29405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:55.314354 29405 catalog_manager.cc:1347] Generated new cluster ID: 06610d886b0147dabb2b5e95f92a430d
I20260620 07:10:55.314412 29405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:55.344292 29405 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:55.345271 29405 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:55.352464 29405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 720f91f2d563468a86377848d68c3d3a: Generated new TSK 0
I20260620 07:10:55.353118 29405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:55.359056 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29318
W20260620 07:10:55.363577 29303 connection.cc:535] client connection to 127.0.0.1:37651 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37651): Connection reset by peer (error 104)
2026-06-20T07:10:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:55 dist-test-slave-wjw3 krb5kdc[29417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:55 dist-test-slave-wjw3 krb5kdc[29417](info): set up 2 sockets
Jun 20 07:10:55 dist-test-slave-wjw3 krb5kdc[29417](info): commencing operation
krb5kdc: starting...
W20260620 07:10:56.299397 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:56 dist-test-slave-wjw3 krb5kdc[29417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:56.408314 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45061
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:56.512341 29430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:56.512589 29430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:56.512629 29430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:56.516544 29430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:56.516598 29430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:56.516616 29430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:56.516630 29430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:56.516651 29430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:56.521713 29430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:56.522630 29430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:56.523563 29430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:56.531265 29438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:56.531339 29430 server_base.cc:1034] running on GCE node
W20260620 07:10:56.531276 29435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:56.531255 29436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:56.532071 29430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:56.533108 29430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:56.534260 29430 hybrid_clock.cc:648] HybridClock initialized: now 1781939456534242 us; error 34 us; skew 500 ppm
Jun 20 07:10:56 dist-test-slave-wjw3 krb5kdc[29417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:56.541260 29430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:56.542081 29430 webserver.cc:460] Webserver started at http://127.0.0.1:45671/ using document root <none> and password file <none>
I20260620 07:10:56.542586 29430 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:56.542650 29430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:56.542840 29430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:56.544804 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "326ea77657ec494da06dad9b2b34e64d"
format_stamp: "Formatted at 2026-06-20 07:10:56 on dist-test-slave-wjw3"
server_key: "56f87a1ce2eb62caac24450a732f7806"
server_key_iv: "6397dc69d85c23d98b6ca28d4134750d"
server_key_version: "encryptionkey@0"
I20260620 07:10:56.545310 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "326ea77657ec494da06dad9b2b34e64d"
format_stamp: "Formatted at 2026-06-20 07:10:56 on dist-test-slave-wjw3"
server_key: "56f87a1ce2eb62caac24450a732f7806"
server_key_iv: "6397dc69d85c23d98b6ca28d4134750d"
server_key_version: "encryptionkey@0"
I20260620 07:10:56.548732 29430 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:56.550993 29445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:56.552258 29430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:56.552378 29430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "326ea77657ec494da06dad9b2b34e64d"
format_stamp: "Formatted at 2026-06-20 07:10:56 on dist-test-slave-wjw3"
server_key: "56f87a1ce2eb62caac24450a732f7806"
server_key_iv: "6397dc69d85c23d98b6ca28d4134750d"
server_key_version: "encryptionkey@0"
I20260620 07:10:56.552474 29430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:56.556807 29430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:56.559791 29430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:56.559947 29430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:56.566416 29430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45061
I20260620 07:10:56.566442 29497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45061 every 8 connection(s)
I20260620 07:10:56.567277 29430 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:56.569880 29498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:56.572872 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29430
I20260620 07:10:56.572997 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:56.573246 8288 external_mini_cluster.cc:1428] Setting key 7cd25036c8c148e0860e6f205905522c
I20260620 07:10:56.575565 29498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d: Bootstrap starting.
I20260620 07:10:56.577859 29498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:56.578243 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:56.574528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:46673 (local address 127.0.0.1:45061)
0620 07:10:56.574960 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:56.574981 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:56.575015 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:10:56.575672 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:56.575696 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:56.575990 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:56.576300 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:56.577090 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:56.577095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:56.577106 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:56.577116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:56.577167 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:56.577186 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:56.577304 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:56.577375 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 07:10:56.577536 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":137,"threads_started":1}
I20260620 07:10:56.578591 29498 log.cc:824] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:56.580452 29498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d: No bootstrap required, opened a new log
I20260620 07:10:56.580703 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:56.583932 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:56.582476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:42433 (local address 127.0.0.1:45061)
0620 07:10:56.582572 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:56.582575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:56.582743 (+ 168us) server_negotiation.cc:407] Connection header received
0620 07:10:56.582839 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:56.582841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:56.582871 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:56.582929 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:56.583607 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:56.583612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:56.583614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:56.583625 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:56.583648 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:56.583658 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:56.583721 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:56.583798 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 07:10:56.583831 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 07:10:56.584189 29498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } }
I20260620 07:10:56.584424 29498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:56.584460 29498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326ea77657ec494da06dad9b2b34e64d, State: Initialized, Role: FOLLOWER
I20260620 07:10:56.584869 29498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } }
I20260620 07:10:56.585001 29498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:56.585062 29498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:56.585148 29498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:56.585938 29498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } }
I20260620 07:10:56.586233 29498 leader_election.cc:302] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 326ea77657ec494da06dad9b2b34e64d; no voters:
I20260620 07:10:56.586488 29498 leader_election.cc:288] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:56.586568 29508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:56.586777 29508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [term 1 LEADER]: Becoming Leader. State: Replica: 326ea77657ec494da06dad9b2b34e64d, State: Running, Role: LEADER
I20260620 07:10:56.587106 29508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } }
I20260620 07:10:56.587639 29498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:56.588703 29509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "326ea77657ec494da06dad9b2b34e64d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } } }
I20260620 07:10:56.588833 29509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:56.589169 29510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 326ea77657ec494da06dad9b2b34e64d. Latest consensus state: current_term: 1 leader_uuid: "326ea77657ec494da06dad9b2b34e64d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326ea77657ec494da06dad9b2b34e64d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45061 } } }
I20260620 07:10:56.589236 29517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:56.589867 29510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:56.592217 29517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:56.597213 29517 catalog_manager.cc:1347] Generated new cluster ID: ff545f2b417e45ccaf26d5c7e31b3782
I20260620 07:10:56.597271 29517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:56.634871 29517 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:56.635213 29517 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:56.644151 29517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 326ea77657ec494da06dad9b2b34e64d: Generated new TSK 0
I20260620 07:10:56.644925 29517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:56.651170 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29430
2026-06-20T07:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:56 dist-test-slave-wjw3 krb5kdc[29529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:56 dist-test-slave-wjw3 krb5kdc[29529](info): set up 2 sockets
Jun 20 07:10:56 dist-test-slave-wjw3 krb5kdc[29529](info): commencing operation
krb5kdc: starting...
W20260620 07:10:57.603411 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:57.712325 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:57.816761 29542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:57.816999 29542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:57.817039 29542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:57.820919 29542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:57.820964 29542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:57.820983 29542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:57.821012 29542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:57.821038 29542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:57.826041 29542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:57.826943 29542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:57.827888 29542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:57.835459 29550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:57.835490 29547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:57.835614 29542 server_base.cc:1034] running on GCE node
W20260620 07:10:57.835490 29548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:57.836223 29542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:57.837198 29542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:57.838348 29542 hybrid_clock.cc:648] HybridClock initialized: now 1781939457838336 us; error 25 us; skew 500 ppm
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:57.840793 29542 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:57.841555 29542 webserver.cc:460] Webserver started at http://127.0.0.1:32811/ using document root <none> and password file <none>
I20260620 07:10:57.842033 29542 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:57.842092 29542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:57.842267 29542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:57.844177 29542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "78e6a99fb7044f4597f442c8f1417b4a"
format_stamp: "Formatted at 2026-06-20 07:10:57 on dist-test-slave-wjw3"
server_key: "11f7ae87a9218a0142821b6e67ec6de1"
server_key_iv: "2abcd69aecacaf70d4c48d66656920f0"
server_key_version: "encryptionkey@0"
I20260620 07:10:57.844677 29542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "78e6a99fb7044f4597f442c8f1417b4a"
format_stamp: "Formatted at 2026-06-20 07:10:57 on dist-test-slave-wjw3"
server_key: "11f7ae87a9218a0142821b6e67ec6de1"
server_key_iv: "2abcd69aecacaf70d4c48d66656920f0"
server_key_version: "encryptionkey@0"
I20260620 07:10:57.847978 29542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:10:57.850106 29557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:57.851226 29542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:57.851325 29542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "78e6a99fb7044f4597f442c8f1417b4a"
format_stamp: "Formatted at 2026-06-20 07:10:57 on dist-test-slave-wjw3"
server_key: "11f7ae87a9218a0142821b6e67ec6de1"
server_key_iv: "2abcd69aecacaf70d4c48d66656920f0"
server_key_version: "encryptionkey@0"
I20260620 07:10:57.851408 29542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:57.864800 29542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:57.867278 29542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:57.867445 29542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:57.873289 29542 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37079
I20260620 07:10:57.873334 29609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37079 every 8 connection(s)
I20260620 07:10:57.874145 29542 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:57.876771 29610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:57.876767 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29542
I20260620 07:10:57.876875 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:57.877115 8288 external_mini_cluster.cc:1428] Setting key 3bdd84ad830ba02b68a831444dc647cb
I20260620 07:10:57.882977 29610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: Bootstrap starting.
I20260620 07:10:57.885529 29610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: Neither blocks nor log segments found. Creating new log.
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 07:10:57.886227 29610 log.cc:824] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:57.887964 29610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: No bootstrap required, opened a new log
I20260620 07:10:57.890457 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:57.878353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:33953 (local address 127.0.0.1:37079)
0620 07:10:57.878742 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:57.878757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:57.878791 (+ 34us) server_negotiation.cc:407] Connection header received
0620 07:10:57.879614 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:57.879632 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:57.879953 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:57.880219 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:57.881069 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:57.881993 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:57.883114 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:57.883601 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:57.883713 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:57.886322 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:57.886326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:57.886342 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:57.886371 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:57.887890 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:57.888357 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:57.888362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:57.888366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:57.888469 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:57.888776 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:57.888779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:57.888780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:57.889130 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:57.889314 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:57.889613 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 07:10:57.889800 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":125,"threads_started":1}
I20260620 07:10:57.891414 29610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260620 07:10:57.891683 29610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:57.891736 29610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78e6a99fb7044f4597f442c8f1417b4a, State: Initialized, Role: FOLLOWER
I20260620 07:10:57.892154 29610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260620 07:10:57.892304 29610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:57.892359 29610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:57.892426 29610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:57.893285 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:57.893370 29610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260620 07:10:57.893643 29610 leader_election.cc:302] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78e6a99fb7044f4597f442c8f1417b4a; no voters:
I20260620 07:10:57.893867 29610 leader_election.cc:288] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:57.893970 29615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:57.894173 29615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [term 1 LEADER]: Becoming Leader. State: Replica: 78e6a99fb7044f4597f442c8f1417b4a, State: Running, Role: LEADER
I20260620 07:10:57.894521 29615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } }
I20260620 07:10:57.894948 29610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:57.896133 29616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78e6a99fb7044f4597f442c8f1417b4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } } }
I20260620 07:10:57.896247 29616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:57.896452 29616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78e6a99fb7044f4597f442c8f1417b4a. Latest consensus state: current_term: 1 leader_uuid: "78e6a99fb7044f4597f442c8f1417b4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78e6a99fb7044f4597f442c8f1417b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37079 } } }
I20260620 07:10:57.896518 29616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a [sys.catalog]: This master's current role is: LEADER
W20260620 07:10:57.899868 29635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:10:57.899967 29635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:10:57.900157 29629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:57.900996 29629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:57.903720 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:57.895232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:51169 (local address 127.0.0.1:37079)
0620 07:10:57.895401 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:57.895405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:57.895629 (+ 224us) server_negotiation.cc:407] Connection header received
0620 07:10:57.895766 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:57.895769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:57.895812 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:57.895859 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:57.896717 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:57.897427 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:57.898252 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:57.898550 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:57.898637 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:57.900139 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:57.900143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:57.900145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:10:57.900168 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:57.901666 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:57.902182 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:57.902186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:57.902187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:57.902264 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:10:57.902651 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:10:57.902654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:10:57.902656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:10:57.902851 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:57.902977 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:57.903554 (+ 577us) server_negotiation.cc:299] Negotiation successful
0620 07:10:57.903624 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260620 07:10:57.906947 29629 catalog_manager.cc:1347] Generated new cluster ID: dc96242651d340009962c4db01ce5ba2
I20260620 07:10:57.907008 29629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:57.924261 29629 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:57.925139 29629 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:57.929971 29629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78e6a99fb7044f4597f442c8f1417b4a: Generated new TSK 0
I20260620 07:10:57.930569 29629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:57.939612 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29542
W20260620 07:10:57.944010 29527 connection.cc:535] client connection to 127.0.0.1:37079 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37079): Connection reset by peer (error 104)
2026-06-20T07:10:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29641](info): set up 2 sockets
Jun 20 07:10:57 dist-test-slave-wjw3 krb5kdc[29641](info): commencing operation
krb5kdc: starting...
W20260620 07:10:58.891362 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:10:58 dist-test-slave-wjw3 krb5kdc[29641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:10:58.998275 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:10:59.105793 29654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:10:59.106038 29654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:10:59.106096 29654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:10:59.110021 29654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:10:59.110073 29654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:10:59.110098 29654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:10:59.110131 29654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:10:59.110160 29654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:10:59.115161 29654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:10:59.116125 29654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:10:59.117041 29654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:10:59.124481 29659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:59.124488 29660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:10:59.124615 29662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:10:59.124683 29654 server_base.cc:1034] running on GCE node
I20260620 07:10:59.125146 29654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:10:59.126051 29654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:10:59.127202 29654 hybrid_clock.cc:648] HybridClock initialized: now 1781939459127188 us; error 27 us; skew 500 ppm
Jun 20 07:10:59 dist-test-slave-wjw3 krb5kdc[29641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:10:59.129635 29654 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:10:59.130409 29654 webserver.cc:460] Webserver started at http://127.0.0.1:41331/ using document root <none> and password file <none>
I20260620 07:10:59.130874 29654 fs_manager.cc:359] Metadata directory not provided
I20260620 07:10:59.130926 29654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:10:59.131080 29654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:10:59.132973 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "360da354c85c4ecf8ad750f76107990b"
format_stamp: "Formatted at 2026-06-20 07:10:59 on dist-test-slave-wjw3"
server_key: "0620f3424eebf4770070c1b186a109b8"
server_key_iv: "71125ca5f6c42886cdc2588e70a29d3d"
server_key_version: "encryptionkey@0"
I20260620 07:10:59.133432 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "360da354c85c4ecf8ad750f76107990b"
format_stamp: "Formatted at 2026-06-20 07:10:59 on dist-test-slave-wjw3"
server_key: "0620f3424eebf4770070c1b186a109b8"
server_key_iv: "71125ca5f6c42886cdc2588e70a29d3d"
server_key_version: "encryptionkey@0"
I20260620 07:10:59.136725 29654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:10:59.138855 29669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:10:59.140028 29654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:10:59.140141 29654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "360da354c85c4ecf8ad750f76107990b"
format_stamp: "Formatted at 2026-06-20 07:10:59 on dist-test-slave-wjw3"
server_key: "0620f3424eebf4770070c1b186a109b8"
server_key_iv: "71125ca5f6c42886cdc2588e70a29d3d"
server_key_version: "encryptionkey@0"
I20260620 07:10:59.140229 29654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:10:59.156973 29654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:10:59.159803 29654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:10:59.159986 29654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:10:59.166190 29654 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35559
I20260620 07:10:59.166266 29721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35559 every 8 connection(s)
I20260620 07:10:59.167084 29654 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:10:59.169723 29722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:10:59.173336 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29654
I20260620 07:10:59.173471 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:10:59.173754 8288 external_mini_cluster.cc:1428] Setting key 2c0ad96864c1de5d2a5aeb9bac8b2392
I20260620 07:10:59.175320 29722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b: Bootstrap starting.
I20260620 07:10:59.177415 29722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b: Neither blocks nor log segments found. Creating new log.
I20260620 07:10:59.178090 29722 log.cc:824] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b: Log is configured to *not* fsync() on all Append() calls
I20260620 07:10:59.179875 29722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b: No bootstrap required, opened a new log
I20260620 07:10:59.182191 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:59.175106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:33677 (local address 127.0.0.1:35559)
0620 07:10:59.175554 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:59.175569 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:59.175605 (+ 36us) server_negotiation.cc:407] Connection header received
0620 07:10:59.176235 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:59.176251 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:59.176594 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:59.176901 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:59.177290 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:59.178299 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:59.179183 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:59.179683 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:59.179824 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:59.180645 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:59.180649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:59.180662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:59.181031 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:59.181082 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:59.181104 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:59.181212 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:59.181385 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 07:10:59.181512 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":131,"threads_started":1}
I20260620 07:10:59.183327 29722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260620 07:10:59.183569 29722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:10:59.183624 29722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360da354c85c4ecf8ad750f76107990b, State: Initialized, Role: FOLLOWER
I20260620 07:10:59.184057 29722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260620 07:10:59.184206 29722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:10:59.184269 29722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:10:59.184360 29722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:10:59.184831 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:10:59.185351 29722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260620 07:10:59.185698 29722 leader_election.cc:302] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 360da354c85c4ecf8ad750f76107990b; no voters:
I20260620 07:10:59.186002 29722 leader_election.cc:288] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:10:59.186087 29727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:10:59.186331 29727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [term 1 LEADER]: Becoming Leader. State: Replica: 360da354c85c4ecf8ad750f76107990b, State: Running, Role: LEADER
I20260620 07:10:59.186667 29727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } }
I20260620 07:10:59.187258 29722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:10:59.187788 29732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 360da354c85c4ecf8ad750f76107990b. Latest consensus state: current_term: 1 leader_uuid: "360da354c85c4ecf8ad750f76107990b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } } }
I20260620 07:10:59.187919 29732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:59.188504 29733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "360da354c85c4ecf8ad750f76107990b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360da354c85c4ecf8ad750f76107990b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35559 } } }
I20260620 07:10:59.188584 29733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b [sys.catalog]: This master's current role is: LEADER
I20260620 07:10:59.189393 29738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:10:59.191582 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:10:59.186856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:42999 (local address 127.0.0.1:35559)
0620 07:10:59.186963 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:10:59.186967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:10:59.187528 (+ 561us) server_negotiation.cc:407] Connection header received
0620 07:10:59.187699 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:10:59.187702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:10:59.187745 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:10:59.187792 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:10:59.188183 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:59.188924 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:59.189973 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:10:59.190194 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:10:59.190274 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:10:59.191079 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:10:59.191083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:10:59.191086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:10:59.191098 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:10:59.191124 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:10:59.191136 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:10:59.191190 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 07:10:59.191404 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 07:10:59.191458 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:10:59.192186 29738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:10:59.196938 29738 catalog_manager.cc:1347] Generated new cluster ID: 3b05aa4ff85f43c9bdc74e263874046b
I20260620 07:10:59.196995 29738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:10:59.207563 29738 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:10:59.208345 29738 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:10:59.213455 29738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 360da354c85c4ecf8ad750f76107990b: Generated new TSK 0
I20260620 07:10:59.214138 29738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:10:59.218492 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29654
W20260620 07:10:59.222843 29639 connection.cc:535] client connection to 127.0.0.1:35559 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35559): Connection reset by peer (error 104)
2026-06-20T07:10:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:10:59 dist-test-slave-wjw3 krb5kdc[29753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:10:59 dist-test-slave-wjw3 krb5kdc[29753](info): set up 2 sockets
Jun 20 07:10:59 dist-test-slave-wjw3 krb5kdc[29753](info): commencing operation
krb5kdc: starting...
W20260620 07:11:00.163403 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:00 dist-test-slave-wjw3 krb5kdc[29753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 07:11:00.270066 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:00.372766 29766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:00.373018 29766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:00.373055 29766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:00.376973 29766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:00.377018 29766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:00.377036 29766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:00.377050 29766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:00.377064 29766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:00.381987 29766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:00.382860 29766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:00.383769 29766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:00.391497 29771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:00.391635 29766 server_base.cc:1034] running on GCE node
W20260620 07:11:00.391530 29772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:00.391511 29774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:00.392241 29766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:00.393185 29766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:00.394332 29766 hybrid_clock.cc:648] HybridClock initialized: now 1781939460394311 us; error 34 us; skew 500 ppm
Jun 20 07:11:00 dist-test-slave-wjw3 krb5kdc[29753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:00.396795 29766 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 07:11:00.397553 29766 webserver.cc:460] Webserver started at http://127.0.0.1:36857/ using document root <none> and password file <none>
I20260620 07:11:00.398005 29766 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:00.398056 29766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:00.398205 29766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:00.400028 29766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "db1b81efd9014628ab2d09cec410749d"
format_stamp: "Formatted at 2026-06-20 07:11:00 on dist-test-slave-wjw3"
server_key: "9a5900ba216d38010142cbca5584efaf"
server_key_iv: "65382b0856822a5d5a94eb4797a13bd2"
server_key_version: "encryptionkey@0"
I20260620 07:11:00.400532 29766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "db1b81efd9014628ab2d09cec410749d"
format_stamp: "Formatted at 2026-06-20 07:11:00 on dist-test-slave-wjw3"
server_key: "9a5900ba216d38010142cbca5584efaf"
server_key_iv: "65382b0856822a5d5a94eb4797a13bd2"
server_key_version: "encryptionkey@0"
I20260620 07:11:00.403820 29766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 07:11:00.405936 29781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:00.407021 29766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:00.407121 29766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "db1b81efd9014628ab2d09cec410749d"
format_stamp: "Formatted at 2026-06-20 07:11:00 on dist-test-slave-wjw3"
server_key: "9a5900ba216d38010142cbca5584efaf"
server_key_iv: "65382b0856822a5d5a94eb4797a13bd2"
server_key_version: "encryptionkey@0"
I20260620 07:11:00.407197 29766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:00.411181 29766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:00.413889 29766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:00.414016 29766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:00.420281 29766 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33821
I20260620 07:11:00.420297 29833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33821 every 8 connection(s)
I20260620 07:11:00.421067 29766 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:00.423542 29834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:00.424666 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29766
I20260620 07:11:00.424770 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:00.425005 8288 external_mini_cluster.cc:1428] Setting key b0732a900b47122b2b68e1e07faec585
I20260620 07:11:00.429041 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d: Bootstrap starting.
I20260620 07:11:00.429613 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:00.426196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:33293 (local address 127.0.0.1:33821)
0620 07:11:00.426620 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:00.426633 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:00.426664 (+ 31us) server_negotiation.cc:407] Connection header received
0620 07:11:00.427241 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:00.427256 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:00.427555 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:00.427828 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:00.428530 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:00.428534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:00.428546 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:00.428556 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:00.428603 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:00.428622 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:00.428735 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:00.428825 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 07:11:00.428965 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
I20260620 07:11:00.431063 29834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:00.431871 29834 log.cc:824] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:00.432031 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:00.433861 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d: No bootstrap required, opened a new log
I20260620 07:11:00.435330 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:00.434014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:35147 (local address 127.0.0.1:33821)
0620 07:11:00.434121 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:00.434125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:00.434304 (+ 179us) server_negotiation.cc:407] Connection header received
0620 07:11:00.434425 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:00.434426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:00.434450 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:00.434511 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:00.435042 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:00.435045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:00.435046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:00.435055 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:00.435076 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:00.435085 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:00.435143 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:00.435222 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 07:11:00.435248 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:11:00.437611 29834 raft_consensus.cc:348] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } }
I20260620 07:11:00.437851 29834 raft_consensus.cc:374] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:00.437903 29834 raft_consensus.cc:729] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1b81efd9014628ab2d09cec410749d, State: Initialized, Role: FOLLOWER
I20260620 07:11:00.438232 29834 consensus_queue.cc:260] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } }
I20260620 07:11:00.438338 29834 raft_consensus.cc:388] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:00.438380 29834 raft_consensus.cc:482] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:00.438442 29834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:00.439145 29834 raft_consensus.cc:504] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } }
I20260620 07:11:00.439414 29834 leader_election.cc:302] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db1b81efd9014628ab2d09cec410749d; no voters:
I20260620 07:11:00.439688 29834 leader_election.cc:288] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:00.439769 29844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:00.439966 29844 raft_consensus.cc:686] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [term 1 LEADER]: Becoming Leader. State: Replica: db1b81efd9014628ab2d09cec410749d, State: Running, Role: LEADER
I20260620 07:11:00.440265 29844 consensus_queue.cc:237] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } }
I20260620 07:11:00.440694 29834 sys_catalog.cc:564] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:00.441565 29846 sys_catalog.cc:455] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [sys.catalog]: SysCatalogTable state changed. Reason: New leader db1b81efd9014628ab2d09cec410749d. Latest consensus state: current_term: 1 leader_uuid: "db1b81efd9014628ab2d09cec410749d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } } }
I20260620 07:11:00.441733 29846 sys_catalog.cc:458] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:00.441776 29845 sys_catalog.cc:455] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db1b81efd9014628ab2d09cec410749d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db1b81efd9014628ab2d09cec410749d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33821 } } }
I20260620 07:11:00.441926 29845 sys_catalog.cc:458] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:00.442139 29853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:00.444609 29853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:00.449060 29853 catalog_manager.cc:1347] Generated new cluster ID: 0b38cd82aa0f4e269dc7ffb8ba7abd6f
I20260620 07:11:00.449115 29853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:00.460405 29853 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:00.460686 29853 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:00.465680 29853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db1b81efd9014628ab2d09cec410749d: Generated new TSK 0
I20260620 07:11:00.466342 29853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:00.473721 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29766
2026-06-20T07:11:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:00 dist-test-slave-wjw3 krb5kdc[29865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:00 dist-test-slave-wjw3 krb5kdc[29865](info): set up 2 sockets
Jun 20 07:11:00 dist-test-slave-wjw3 krb5kdc[29865](info): commencing operation
krb5kdc: starting...
W20260620 07:11:01.415377 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:01.521998 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43897
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:43897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:01.625306 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:01.625543 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:01.625581 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:01.629501 29878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:01.629552 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:01.629570 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:01.629585 29878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:01.629598 29878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:01.634644 29878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:43897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:01.635639 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:01.636650 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:01.644469 29886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:01.644479 29884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:01.644637 29878 server_base.cc:1034] running on GCE node
W20260620 07:11:01.644479 29883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:01.645207 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:01.646212 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:01.647370 29878 hybrid_clock.cc:648] HybridClock initialized: now 1781939461647347 us; error 35 us; skew 500 ppm
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939461, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:01.653880 29878 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:01.654712 29878 webserver.cc:460] Webserver started at http://127.8.24.62:34049/ using document root <none> and password file <none>
I20260620 07:11:01.655205 29878 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:01.655261 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:01.655473 29878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:01.657441 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "252de27097934d8e8c322f57e1ab798b"
format_stamp: "Formatted at 2026-06-20 07:11:01 on dist-test-slave-wjw3"
server_key: "a10a4721f2d4886baa5d1d93fabfcff2"
server_key_iv: "745cd33ec6fe768ffdea7901b5954db2"
server_key_version: "encryptionkey@0"
I20260620 07:11:01.657949 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "252de27097934d8e8c322f57e1ab798b"
format_stamp: "Formatted at 2026-06-20 07:11:01 on dist-test-slave-wjw3"
server_key: "a10a4721f2d4886baa5d1d93fabfcff2"
server_key_iv: "745cd33ec6fe768ffdea7901b5954db2"
server_key_version: "encryptionkey@0"
I20260620 07:11:01.661437 29878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:11:01.663681 29893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:01.664788 29878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:11:01.664901 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "252de27097934d8e8c322f57e1ab798b"
format_stamp: "Formatted at 2026-06-20 07:11:01 on dist-test-slave-wjw3"
server_key: "a10a4721f2d4886baa5d1d93fabfcff2"
server_key_iv: "745cd33ec6fe768ffdea7901b5954db2"
server_key_version: "encryptionkey@0"
I20260620 07:11:01.664991 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:01.686252 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:01.688933 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:01.689092 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:01.695214 29878 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:43897
I20260620 07:11:01.695267 29945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:43897 every 8 connection(s)
I20260620 07:11:01.696090 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:01.697034 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29878
I20260620 07:11:01.697149 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:01.697383 8288 external_mini_cluster.cc:1428] Setting key 8b206d0bd8fea241807737b9d095e5d8
I20260620 07:11:01.699025 29946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:01.705245 29946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b: Bootstrap starting.
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:01.707397 29946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:01.708249 29946 log.cc:824] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:01.710318 29946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b: No bootstrap required, opened a new log
I20260620 07:11:01.711488 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:01.698651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58985 (local address 127.8.24.62:43897)
0620 07:11:01.699031 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:01.699049 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:01.699090 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:11:01.699926 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:01.699949 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:01.700354 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:01.700694 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:01.701389 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:01.702831 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:01.703727 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:01.704406 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:01.704552 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:01.706745 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:01.706752 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:01.706765 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:01.706794 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:01.708572 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:01.709086 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:01.709092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:01.709097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:01.709219 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:01.709491 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:01.709494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:01.709495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:01.709944 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:01.710139 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:01.710400 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 07:11:01.710604 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
I20260620 07:11:01.713861 29946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } }
I20260620 07:11:01.714092 29946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:01.714186 29946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252de27097934d8e8c322f57e1ab798b, State: Initialized, Role: FOLLOWER
I20260620 07:11:01.714562 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:01.714599 29946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } }
I20260620 07:11:01.714746 29946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:01.714814 29946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:01.714903 29946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:01.715760 29946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } }
I20260620 07:11:01.716081 29946 leader_election.cc:302] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 252de27097934d8e8c322f57e1ab798b; no voters:
I20260620 07:11:01.716394 29946 leader_election.cc:288] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:01.716730 29957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:01.716882 29957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [term 1 LEADER]: Becoming Leader. State: Replica: 252de27097934d8e8c322f57e1ab798b, State: Running, Role: LEADER
I20260620 07:11:01.717149 29957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } }
I20260620 07:11:01.717536 29946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:01.718086 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 252de27097934d8e8c322f57e1ab798b. Latest consensus state: current_term: 1 leader_uuid: "252de27097934d8e8c322f57e1ab798b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } } }
I20260620 07:11:01.718245 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:01.718473 29958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "252de27097934d8e8c322f57e1ab798b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "252de27097934d8e8c322f57e1ab798b" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 43897 } } }
I20260620 07:11:01.718567 29958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:01.718640 29964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:01.721571 29964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:01.727113 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:01.716413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37173 (local address 127.8.24.62:43897)
0620 07:11:01.716512 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:01.716515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:01.719546 (+ 3031us) server_negotiation.cc:407] Connection header received
0620 07:11:01.719677 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:01.719680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:01.719725 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:01.719799 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:01.720446 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:01.721149 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:01.722245 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:01.722573 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:01.722670 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:01.724207 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:01.724210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:01.724213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:01.724236 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:01.725604 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:01.726091 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:01.726094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:01.726095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:01.726159 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:01.726442 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:01.726445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:01.726446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:01.726586 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:01.726684 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:01.726923 (+ 239us) server_negotiation.cc:299] Negotiation successful
0620 07:11:01.726996 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 07:11:01.728561 29964 catalog_manager.cc:1347] Generated new cluster ID: be1731989f2e433f8fc767872973cf77
I20260620 07:11:01.728617 29964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:01.741461 29964 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:01.742304 29964 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:01.748495 29964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 252de27097934d8e8c322f57e1ab798b: Generated new TSK 0
I20260620 07:11:01.749094 29964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:01.754742 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29878
W20260620 07:11:01.759179 29863 connection.cc:535] client connection to 127.8.24.62:43897 recv error: Network error: failed to read from TLS socket (remote: 127.8.24.62:43897): Connection reset by peer (error 104)
2026-06-20T07:11:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29977](info): set up 2 sockets
Jun 20 07:11:01 dist-test-slave-wjw3 krb5kdc[29977](info): commencing operation
krb5kdc: starting...
W20260620 07:11:02.699360 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:02 dist-test-slave-wjw3 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:02.805727 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44935
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:02.909858 29990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:02.910091 29990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:02.910145 29990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:02.914069 29990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:02.914124 29990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:02.914150 29990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:02.914171 29990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:02.914199 29990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:02.919147 29990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:02.920115 29990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:02.921087 29990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:11:02.928905 29990 server_base.cc:1034] running on GCE node
W20260620 07:11:02.928824 29996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:02.928874 29995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:02.928824 29998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:02.929572 29990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:02.930601 29990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:02.931752 29990 hybrid_clock.cc:648] HybridClock initialized: now 1781939462931739 us; error 23 us; skew 500 ppm
Jun 20 07:11:02 dist-test-slave-wjw3 krb5kdc[29977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:02.934226 29990 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:02.935032 29990 webserver.cc:460] Webserver started at http://127.8.24.62:40409/ using document root <none> and password file <none>
I20260620 07:11:02.935549 29990 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:02.935613 29990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:02.935822 29990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:02.937752 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "c401228267e64980a799ccd29ca0e0de"
format_stamp: "Formatted at 2026-06-20 07:11:02 on dist-test-slave-wjw3"
server_key: "269346f644f934cfaca01146ed3dc7e6"
server_key_iv: "31ee5e9ed8bbe244d21f397e5e545366"
server_key_version: "encryptionkey@0"
I20260620 07:11:02.938284 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "c401228267e64980a799ccd29ca0e0de"
format_stamp: "Formatted at 2026-06-20 07:11:02 on dist-test-slave-wjw3"
server_key: "269346f644f934cfaca01146ed3dc7e6"
server_key_iv: "31ee5e9ed8bbe244d21f397e5e545366"
server_key_version: "encryptionkey@0"
I20260620 07:11:02.941838 29990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:11:02.944077 30005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:02.945268 29990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:02.945379 29990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "c401228267e64980a799ccd29ca0e0de"
format_stamp: "Formatted at 2026-06-20 07:11:02 on dist-test-slave-wjw3"
server_key: "269346f644f934cfaca01146ed3dc7e6"
server_key_iv: "31ee5e9ed8bbe244d21f397e5e545366"
server_key_version: "encryptionkey@0"
I20260620 07:11:02.945466 29990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:02.973351 29990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:02.976399 29990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:02.976564 29990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:02.982888 29990 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44935
I20260620 07:11:02.982903 30057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44935 every 8 connection(s)
I20260620 07:11:02.983829 29990 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:02.986353 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:02.990520 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 29990
I20260620 07:11:02.990638 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:02.990885 8288 external_mini_cluster.cc:1428] Setting key 0cb96cdc6ed31ee5868a3b6cc717edcc
I20260620 07:11:02.991544 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de: Bootstrap starting.
I20260620 07:11:02.994086 30058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:02.994895 30058 log.cc:824] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:02.996693 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de: No bootstrap required, opened a new log
I20260620 07:11:03.000135 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:02.992081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39699 (local address 127.8.24.62:44935)
0620 07:11:02.992448 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:02.992466 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:02.992505 (+ 39us) server_negotiation.cc:407] Connection header received
0620 07:11:02.993303 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:02.993325 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:02.993718 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:02.994060 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:02.994460 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:02.995769 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:02.996665 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:02.997320 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:02.997477 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:02.998182 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:02.998188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:02.998203 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:02.998664 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:02.998729 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:02.998757 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:02.998882 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:02.999087 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 07:11:02.999220 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
I20260620 07:11:03.000685 30058 raft_consensus.cc:348] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } }
I20260620 07:11:03.000910 30058 raft_consensus.cc:374] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:03.000973 30058 raft_consensus.cc:729] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c401228267e64980a799ccd29ca0e0de, State: Initialized, Role: FOLLOWER
I20260620 07:11:03.001370 30058 consensus_queue.cc:260] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } }
I20260620 07:11:03.001500 30058 raft_consensus.cc:388] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:03.001560 30058 raft_consensus.cc:482] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:03.001652 30058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:03.002562 30058 raft_consensus.cc:504] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } }
I20260620 07:11:03.002758 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:03.002884 30058 leader_election.cc:302] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c401228267e64980a799ccd29ca0e0de; no voters:
I20260620 07:11:03.003144 30058 leader_election.cc:288] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:03.003242 30063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:03.003459 30063 raft_consensus.cc:686] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [term 1 LEADER]: Becoming Leader. State: Replica: c401228267e64980a799ccd29ca0e0de, State: Running, Role: LEADER
I20260620 07:11:03.003809 30063 consensus_queue.cc:237] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } }
I20260620 07:11:03.004276 30058 sys_catalog.cc:564] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:03.004954 30065 sys_catalog.cc:455] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [sys.catalog]: SysCatalogTable state changed. Reason: New leader c401228267e64980a799ccd29ca0e0de. Latest consensus state: current_term: 1 leader_uuid: "c401228267e64980a799ccd29ca0e0de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } } }
I20260620 07:11:03.005132 30065 sys_catalog.cc:458] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:03.005316 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c401228267e64980a799ccd29ca0e0de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c401228267e64980a799ccd29ca0e0de" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44935 } } }
I20260620 07:11:03.005395 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:03.005545 30076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:03.008463 30076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:03.010794 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:03.006248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50191 (local address 127.8.24.62:44935)
0620 07:11:03.006329 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:03.006332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:03.006765 (+ 433us) server_negotiation.cc:407] Connection header received
0620 07:11:03.006905 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:03.006909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:03.006945 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:03.007009 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:03.007248 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:03.008049 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:03.009245 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:03.009527 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:03.009613 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:03.010366 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:03.010370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:03.010373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:03.010386 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:03.010412 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:03.010425 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:03.010515 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:03.010634 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 07:11:03.010666 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 07:11:03.013571 30076 catalog_manager.cc:1347] Generated new cluster ID: bb231adc9c9f499ca6517f7204c95700
I20260620 07:11:03.013633 30076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:03.036518 30076 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:03.037421 30076 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:03.047240 30076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c401228267e64980a799ccd29ca0e0de: Generated new TSK 0
I20260620 07:11:03.047896 30076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:03.056841 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 29990
W20260620 07:11:03.061391 29975 connection.cc:535] client connection to 127.8.24.62:44935 recv error: Network error: failed to read from TLS socket (remote: 127.8.24.62:44935): Connection reset by peer (error 104)
2026-06-20T07:11:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:03 dist-test-slave-wjw3 krb5kdc[30089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:03 dist-test-slave-wjw3 krb5kdc[30089](info): set up 2 sockets
Jun 20 07:11:03 dist-test-slave-wjw3 krb5kdc[30089](info): commencing operation
krb5kdc: starting...
W20260620 07:11:04.007510 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:04 dist-test-slave-wjw3 krb5kdc[30089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:04.116526 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46795
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:46795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:04.219488 30102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:04.219729 30102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:04.219769 30102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:04.223603 30102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:04.223652 30102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:04.223670 30102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:04.223690 30102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:04.223707 30102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:04.228713 30102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:46795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:04.229638 30102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:04.230561 30102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:04.238232 30110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:04.238348 30102 server_base.cc:1034] running on GCE node
W20260620 07:11:04.238293 30107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:04.238245 30108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:04.239054 30102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:04.240134 30102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:04.241312 30102 hybrid_clock.cc:648] HybridClock initialized: now 1781939464241293 us; error 27 us; skew 500 ppm
Jun 20 07:11:04 dist-test-slave-wjw3 krb5kdc[30089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939464, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:04.243878 30102 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:04.244640 30102 webserver.cc:460] Webserver started at http://127.8.24.62:36157/ using document root <none> and password file <none>
I20260620 07:11:04.245123 30102 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:04.245178 30102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:04.245360 30102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:04.247231 30102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "da0cebe0433b4a78b3049391ac540b70"
format_stamp: "Formatted at 2026-06-20 07:11:04 on dist-test-slave-wjw3"
server_key: "950d9f968a8ad798074a7fc339c3363a"
server_key_iv: "3ff39b9c1de4ba51567d5fd25acd0d68"
server_key_version: "encryptionkey@0"
I20260620 07:11:04.247751 30102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "da0cebe0433b4a78b3049391ac540b70"
format_stamp: "Formatted at 2026-06-20 07:11:04 on dist-test-slave-wjw3"
server_key: "950d9f968a8ad798074a7fc339c3363a"
server_key_iv: "3ff39b9c1de4ba51567d5fd25acd0d68"
server_key_version: "encryptionkey@0"
I20260620 07:11:04.251067 30102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:11:04.253278 30117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:04.254386 30102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:04.254480 30102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "da0cebe0433b4a78b3049391ac540b70"
format_stamp: "Formatted at 2026-06-20 07:11:04 on dist-test-slave-wjw3"
server_key: "950d9f968a8ad798074a7fc339c3363a"
server_key_iv: "3ff39b9c1de4ba51567d5fd25acd0d68"
server_key_version: "encryptionkey@0"
I20260620 07:11:04.254571 30102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:04.258610 30102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:04.261127 30102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:04.261266 30102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:04.267194 30102 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:46795
I20260620 07:11:04.267238 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:46795 every 8 connection(s)
I20260620 07:11:04.268079 30102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:04.270596 30170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:04.270800 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30102
I20260620 07:11:04.270917 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:04.271160 8288 external_mini_cluster.cc:1428] Setting key bf27b5bca0a0fdb22d6055e913e91c10
I20260620 07:11:04.276273 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:04.272408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48259 (local address 127.8.24.62:46795)
0620 07:11:04.272821 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:04.272843 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:04.272884 (+ 41us) server_negotiation.cc:407] Connection header received
0620 07:11:04.273605 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:04.273627 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:04.273965 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:04.274337 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:04.274982 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:04.274987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:04.275000 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:04.275011 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:04.275069 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:04.275095 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:04.275220 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:04.275284 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 07:11:04.275431 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":107,"threads_started":1}
I20260620 07:11:04.277226 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70: Bootstrap starting.
I20260620 07:11:04.278869 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:04.279616 30170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:04.280441 30170 log.cc:824] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:04.282325 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70: No bootstrap required, opened a new log
I20260620 07:11:04.282366 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:04.280949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43529 (local address 127.8.24.62:46795)
0620 07:11:04.281059 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:04.281063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:04.281214 (+ 151us) server_negotiation.cc:407] Connection header received
0620 07:11:04.281306 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:04.281309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:04.281337 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:04.281410 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:04.282045 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:04.282054 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:04.282057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:04.282068 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:04.282092 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:04.282105 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:04.282185 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:04.282233 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 07:11:04.282264 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 07:11:04.285593 30170 raft_consensus.cc:348] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } }
I20260620 07:11:04.285812 30170 raft_consensus.cc:374] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:04.285844 30170 raft_consensus.cc:729] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0cebe0433b4a78b3049391ac540b70, State: Initialized, Role: FOLLOWER
I20260620 07:11:04.286180 30170 consensus_queue.cc:260] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } }
I20260620 07:11:04.286279 30170 raft_consensus.cc:388] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:04.286332 30170 raft_consensus.cc:482] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:04.286417 30170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:04.287184 30170 raft_consensus.cc:504] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } }
I20260620 07:11:04.287509 30170 leader_election.cc:302] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da0cebe0433b4a78b3049391ac540b70; no voters:
I20260620 07:11:04.287791 30170 leader_election.cc:288] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:04.287904 30180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:04.288100 30180 raft_consensus.cc:686] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [term 1 LEADER]: Becoming Leader. State: Replica: da0cebe0433b4a78b3049391ac540b70, State: Running, Role: LEADER
I20260620 07:11:04.288432 30180 consensus_queue.cc:237] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } }
I20260620 07:11:04.288906 30170 sys_catalog.cc:564] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:04.289996 30182 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da0cebe0433b4a78b3049391ac540b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } } }
I20260620 07:11:04.290007 30181 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da0cebe0433b4a78b3049391ac540b70. Latest consensus state: current_term: 1 leader_uuid: "da0cebe0433b4a78b3049391ac540b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da0cebe0433b4a78b3049391ac540b70" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 46795 } } }
I20260620 07:11:04.290159 30182 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:04.290176 30181 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:04.291082 30190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:04.293614 30190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:04.298609 30190 catalog_manager.cc:1347] Generated new cluster ID: c3ffffbd0ec34d39ba4bbf26038b9f3b
I20260620 07:11:04.298671 30190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:04.306195 30190 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:04.306463 30190 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:04.320611 30190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da0cebe0433b4a78b3049391ac540b70: Generated new TSK 0
I20260620 07:11:04.321220 30190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:04.326118 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30102
2026-06-20T07:11:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:04 dist-test-slave-wjw3 krb5kdc[30201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:04 dist-test-slave-wjw3 krb5kdc[30201](info): set up 2 sockets
Jun 20 07:11:04 dist-test-slave-wjw3 krb5kdc[30201](info): commencing operation
krb5kdc: starting...
W20260620 07:11:05.267401 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:05.373373 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36515
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:44069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:05.477573 30214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:05.477804 30214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:05.477857 30214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:05.481743 30214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:05.481791 30214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:05.481809 30214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:05.481837 30214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:05.481863 30214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:05.486831 30214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:44069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:36515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:05.487763 30214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:05.488664 30214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:05.496101 30222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:05.496161 30219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:05.496120 30220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:05.496342 30214 server_base.cc:1034] running on GCE node
I20260620 07:11:05.496884 30214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:05.497846 30214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:05.498996 30214 hybrid_clock.cc:648] HybridClock initialized: now 1781939465498983 us; error 25 us; skew 500 ppm
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939465, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:05.501513 30214 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:05.502267 30214 webserver.cc:460] Webserver started at http://127.8.24.62:44125/ using document root <none> and password file <none>
I20260620 07:11:05.502740 30214 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:05.502792 30214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:05.502948 30214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:05.504861 30214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "ae06a2b0559644d6a53491ac9e730330"
format_stamp: "Formatted at 2026-06-20 07:11:05 on dist-test-slave-wjw3"
server_key: "f537fa4ff46c04967d5490759627e515"
server_key_iv: "a0aa2f8b8c3ab9bf100340bbeb64f1d6"
server_key_version: "encryptionkey@0"
I20260620 07:11:05.505354 30214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "ae06a2b0559644d6a53491ac9e730330"
format_stamp: "Formatted at 2026-06-20 07:11:05 on dist-test-slave-wjw3"
server_key: "f537fa4ff46c04967d5490759627e515"
server_key_iv: "a0aa2f8b8c3ab9bf100340bbeb64f1d6"
server_key_version: "encryptionkey@0"
I20260620 07:11:05.508783 30214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:11:05.510913 30229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:05.512089 30214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:05.512198 30214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "ae06a2b0559644d6a53491ac9e730330"
format_stamp: "Formatted at 2026-06-20 07:11:05 on dist-test-slave-wjw3"
server_key: "f537fa4ff46c04967d5490759627e515"
server_key_iv: "a0aa2f8b8c3ab9bf100340bbeb64f1d6"
server_key_version: "encryptionkey@0"
I20260620 07:11:05.512286 30214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:05.523952 30214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:05.526736 30214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:05.526895 30214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:05.533113 30214 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:36515
I20260620 07:11:05.533149 30281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:36515 every 8 connection(s)
I20260620 07:11:05.533959 30214 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:05.536459 30282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:05.538110 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30214
I20260620 07:11:05.538228 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:05.538482 8288 external_mini_cluster.cc:1428] Setting key df1dd065de462ebc577eba5fbc0dcf3f
I20260620 07:11:05.542651 30282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: Bootstrap starting.
I20260620 07:11:05.545075 30282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:05.545840 30282 log.cc:824] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: Log is configured to *not* fsync() on all Append() calls
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:05.547523 30282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: No bootstrap required, opened a new log
I20260620 07:11:05.551698 30282 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } }
I20260620 07:11:05.551946 30282 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:05.552008 30282 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae06a2b0559644d6a53491ac9e730330, State: Initialized, Role: FOLLOWER
I20260620 07:11:05.552394 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:05.539744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37363 (local address 127.8.24.62:36515)
0620 07:11:05.540108 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:05.540126 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:05.540165 (+ 39us) server_negotiation.cc:407] Connection header received
0620 07:11:05.540888 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:05.540911 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:05.541458 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:05.541819 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:05.542442 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:05.543873 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:05.544701 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:05.545362 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:05.545517 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:05.547733 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:05.547738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:05.547753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:05.547783 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:05.549529 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:05.549905 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:05.549911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:05.549917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:05.550047 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:05.550259 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:05.550262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:05.550263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:05.550745 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:05.550945 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:05.551193 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 07:11:05.551497 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":92,"threads_started":1}
I20260620 07:11:05.552459 30282 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } }
I20260620 07:11:05.552623 30282 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:05.552691 30282 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:05.552794 30282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:05.553679 30282 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } }
I20260620 07:11:05.554018 30282 leader_election.cc:302] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae06a2b0559644d6a53491ac9e730330; no voters:
I20260620 07:11:05.554314 30282 leader_election.cc:288] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:05.554389 30287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:05.554601 30287 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [term 1 LEADER]: Becoming Leader. State: Replica: ae06a2b0559644d6a53491ac9e730330, State: Running, Role: LEADER
I20260620 07:11:05.554920 30287 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } }
I20260620 07:11:05.555168 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:05.555572 30282 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:05.557381 30289 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae06a2b0559644d6a53491ac9e730330. Latest consensus state: current_term: 1 leader_uuid: "ae06a2b0559644d6a53491ac9e730330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } } }
I20260620 07:11:05.557505 30289 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:05.557914 30288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae06a2b0559644d6a53491ac9e730330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae06a2b0559644d6a53491ac9e730330" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 36515 } } }
I20260620 07:11:05.558025 30288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330 [sys.catalog]: This master's current role is: LEADER
W20260620 07:11:05.560416 30306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:05.560515 30306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:05.560576 30307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:05.561363 30307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:05.564343 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:05.557492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57335 (local address 127.8.24.62:36515)
0620 07:11:05.557624 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:05.557628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:05.557775 (+ 147us) server_negotiation.cc:407] Connection header received
0620 07:11:05.557946 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:05.557948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:05.558009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:05.558056 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:05.558708 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:05.559287 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:05.560106 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:05.560330 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:05.560410 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:05.561565 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:05.561568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:05.561570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:05.561589 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:05.562761 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:05.563134 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:05.563137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:05.563139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:05.563216 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:05.563558 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:05.563561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:05.563563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:05.563753 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:05.563866 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:05.564100 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 07:11:05.564200 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 07:11:05.566459 30307 catalog_manager.cc:1347] Generated new cluster ID: 11a34efb7a5248a2aa27e2499b48ff06
I20260620 07:11:05.566533 30307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:05.577903 30307 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:05.578902 30307 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:05.585342 30307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae06a2b0559644d6a53491ac9e730330: Generated new TSK 0
I20260620 07:11:05.585898 30307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:05.596751 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30214
2026-06-20T07:11:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30313](info): set up 2 sockets
Jun 20 07:11:05 dist-test-slave-wjw3 krb5kdc[30313](info): commencing operation
krb5kdc: starting...
W20260620 07:11:06.535415 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:06 dist-test-slave-wjw3 krb5kdc[30313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:06.641726 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44907
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:44907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:06.743358 30326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:06.743624 30326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:06.743664 30326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:06.747581 30326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:06.747632 30326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:06.747649 30326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:06.747663 30326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:06.747676 30326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:06.753243 30326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:44907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:44907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:06.754344 30326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:06.755297 30326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:06.763103 30331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:06.763113 30334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:06.763118 30332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:06.763149 30326 server_base.cc:1034] running on GCE node
I20260620 07:11:06.763845 30326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:06.764853 30326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:06.766014 30326 hybrid_clock.cc:648] HybridClock initialized: now 1781939466766000 us; error 30 us; skew 500 ppm
Jun 20 07:11:06 dist-test-slave-wjw3 krb5kdc[30313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939466, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:06.772791 30326 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:06.773618 30326 webserver.cc:460] Webserver started at http://127.8.24.62:45973/ using document root <none> and password file <none>
I20260620 07:11:06.774115 30326 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:06.774176 30326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:06.774348 30326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:06.776296 30326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "cc6a36f45ab647268777146b03c6e829"
format_stamp: "Formatted at 2026-06-20 07:11:06 on dist-test-slave-wjw3"
server_key: "f034b34b35da89fa4166ba801c799dd7"
server_key_iv: "4546c8bcb305c2f6d0d97d0943e32e9b"
server_key_version: "encryptionkey@0"
I20260620 07:11:06.776793 30326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "cc6a36f45ab647268777146b03c6e829"
format_stamp: "Formatted at 2026-06-20 07:11:06 on dist-test-slave-wjw3"
server_key: "f034b34b35da89fa4166ba801c799dd7"
server_key_iv: "4546c8bcb305c2f6d0d97d0943e32e9b"
server_key_version: "encryptionkey@0"
I20260620 07:11:06.780161 30326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 07:11:06.782258 30341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:06.783355 30326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 07:11:06.783486 30326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "cc6a36f45ab647268777146b03c6e829"
format_stamp: "Formatted at 2026-06-20 07:11:06 on dist-test-slave-wjw3"
server_key: "f034b34b35da89fa4166ba801c799dd7"
server_key_iv: "4546c8bcb305c2f6d0d97d0943e32e9b"
server_key_version: "encryptionkey@0"
I20260620 07:11:06.783569 30326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:06.798424 30326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:06.801172 30326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:06.801318 30326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:06.807601 30326 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:44907
I20260620 07:11:06.807611 30393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:44907 every 8 connection(s)
I20260620 07:11:06.808418 30326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:06.810885 30394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:06.815924 30394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: Bootstrap starting.
I20260620 07:11:06.816634 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30326
I20260620 07:11:06.816756 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:06.817045 8288 external_mini_cluster.cc:1428] Setting key da1e99611ff0a3d06b4c90aa3653b7fd
I20260620 07:11:06.818507 30394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:06.819243 30394 log.cc:824] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:06.821846 30394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: No bootstrap required, opened a new log
I20260620 07:11:06.824765 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:06.818072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43711 (local address 127.8.24.62:44907)
0620 07:11:06.818421 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:06.818435 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:06.818467 (+ 32us) server_negotiation.cc:407] Connection header received
0620 07:11:06.819073 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:06.819088 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:06.819505 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:06.819789 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:06.820171 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:06.821082 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:06.821968 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:06.822405 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:06.822549 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:06.823251 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:06.823256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:06.823266 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:06.823645 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:06.823694 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:06.823716 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:06.823808 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:06.823965 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 07:11:06.824141 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":93,"threads_started":1}
I20260620 07:11:06.825202 30394 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } }
I20260620 07:11:06.825404 30394 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:06.825487 30394 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc6a36f45ab647268777146b03c6e829, State: Initialized, Role: FOLLOWER
I20260620 07:11:06.825863 30394 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } }
I20260620 07:11:06.825981 30394 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:06.826040 30394 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:06.826130 30394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:06.827010 30394 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } }
I20260620 07:11:06.827267 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:06.827344 30394 leader_election.cc:302] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc6a36f45ab647268777146b03c6e829; no voters:
I20260620 07:11:06.827651 30394 leader_election.cc:288] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:06.827740 30399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:06.827930 30399 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [term 1 LEADER]: Becoming Leader. State: Replica: cc6a36f45ab647268777146b03c6e829, State: Running, Role: LEADER
I20260620 07:11:06.828274 30399 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } }
I20260620 07:11:06.828684 30394 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:06.829876 30400 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc6a36f45ab647268777146b03c6e829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } } }
I20260620 07:11:06.829984 30400 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:06.830339 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc6a36f45ab647268777146b03c6e829. Latest consensus state: current_term: 1 leader_uuid: "cc6a36f45ab647268777146b03c6e829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc6a36f45ab647268777146b03c6e829" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 44907 } } }
I20260620 07:11:06.830422 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829 [sys.catalog]: This master's current role is: LEADER
W20260620 07:11:06.833176 30419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:06.833247 30419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:06.833315 30414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:06.833997 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:06.829466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46529 (local address 127.8.24.62:44907)
0620 07:11:06.829558 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:06.829561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:06.829572 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:11:06.829603 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:06.829606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:06.829641 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:06.829702 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:06.831033 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:06.831624 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:06.832630 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:06.832860 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:06.832944 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:06.833653 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:06.833656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:06.833658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:06.833668 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:06.833691 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:06.833700 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:06.833752 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:06.833855 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 07:11:06.833893 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 07:11:06.834136 30414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:06.838935 30414 catalog_manager.cc:1347] Generated new cluster ID: c567a37ac308426693408fe37eabcd86
I20260620 07:11:06.838984 30414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:06.859057 30414 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:06.859941 30414 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:06.872460 30414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc6a36f45ab647268777146b03c6e829: Generated new TSK 0
I20260620 07:11:06.873222 30414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:06.887056 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30326
2026-06-20T07:11:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:06 dist-test-slave-wjw3 krb5kdc[30425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:06 dist-test-slave-wjw3 krb5kdc[30425](info): set up 2 sockets
Jun 20 07:11:06 dist-test-slave-wjw3 krb5kdc[30425](info): commencing operation
krb5kdc: starting...
W20260620 07:11:07.831358 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:07 dist-test-slave-wjw3 krb5kdc[30425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:07.935853 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34631
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:35645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:34631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:08.037961 30438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:08.038177 30438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:08.038231 30438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:08.042065 30438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:08.042116 30438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:08.042143 30438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:08.042165 30438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:08.042193 30438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:08.047076 30438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:35645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:34631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:34631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:08.047945 30438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:08.048806 30438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:08.056308 30443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:08.056295 30444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:08.056313 30446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:08.056493 30438 server_base.cc:1034] running on GCE node
I20260620 07:11:08.057065 30438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:08.057997 30438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:08.059137 30438 hybrid_clock.cc:648] HybridClock initialized: now 1781939468059117 us; error 33 us; skew 500 ppm
Jun 20 07:11:08 dist-test-slave-wjw3 krb5kdc[30425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939468, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:08.061563 30438 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:08.062335 30438 webserver.cc:460] Webserver started at http://127.8.24.62:38561/ using document root <none> and password file <none>
I20260620 07:11:08.062811 30438 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:08.062867 30438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:08.063040 30438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:08.064940 30438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "bbb4088863f943fe8991752e901560e1"
format_stamp: "Formatted at 2026-06-20 07:11:08 on dist-test-slave-wjw3"
server_key: "539ee3e435961f3aa15650bc348dd483"
server_key_iv: "cd13b640d8f1192bf14bfee1fb73416a"
server_key_version: "encryptionkey@0"
I20260620 07:11:08.065428 30438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "bbb4088863f943fe8991752e901560e1"
format_stamp: "Formatted at 2026-06-20 07:11:08 on dist-test-slave-wjw3"
server_key: "539ee3e435961f3aa15650bc348dd483"
server_key_iv: "cd13b640d8f1192bf14bfee1fb73416a"
server_key_version: "encryptionkey@0"
I20260620 07:11:08.068717 30438 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 07:11:08.070822 30453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:08.071934 30438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:08.072031 30438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "bbb4088863f943fe8991752e901560e1"
format_stamp: "Formatted at 2026-06-20 07:11:08 on dist-test-slave-wjw3"
server_key: "539ee3e435961f3aa15650bc348dd483"
server_key_iv: "cd13b640d8f1192bf14bfee1fb73416a"
server_key_version: "encryptionkey@0"
I20260620 07:11:08.072116 30438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:08.076268 30438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:08.078672 30438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:08.078855 30438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:08.085044 30438 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:34631
I20260620 07:11:08.085054 30505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:34631 every 8 connection(s)
I20260620 07:11:08.085843 30438 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:08.088292 30506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:08.090479 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30438
I20260620 07:11:08.090595 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:08.090821 8288 external_mini_cluster.cc:1428] Setting key 79b4c9ce1fbc35108b7c7a961ea7fea9
I20260620 07:11:08.093804 30506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1: Bootstrap starting.
I20260620 07:11:08.095887 30509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:08.092096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37125 (local address 127.8.24.62:34631)
0620 07:11:08.092504 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:08.092521 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:08.092561 (+ 40us) server_negotiation.cc:407] Connection header received
0620 07:11:08.093316 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:08.093336 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:08.093663 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:08.094002 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:08.094613 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:08.094618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:08.094631 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:08.094642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:08.094699 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:08.094726 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:08.094851 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:08.094918 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 07:11:08.095066 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":109,"threads_started":1}
I20260620 07:11:08.096262 30506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:08.096969 30506 log.cc:824] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:08.098491 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:08.098837 30506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1: No bootstrap required, opened a new log
I20260620 07:11:08.101941 30509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:08.100533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35283 (local address 127.8.24.62:34631)
0620 07:11:08.100638 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:08.100641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:08.100832 (+ 191us) server_negotiation.cc:407] Connection header received
0620 07:11:08.100906 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:08.100909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:08.100940 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:08.101021 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:08.101607 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:08.101610 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:08.101613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 07:11:08.101625 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:08.101654 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 07:11:08.101667 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:08.101755 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:08.101793 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 07:11:08.101829 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:11:08.102876 30506 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } }
I20260620 07:11:08.103070 30506 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 07:11:08.103114 30506 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb4088863f943fe8991752e901560e1, State: Initialized, Role: FOLLOWER
I20260620 07:11:08.103528 30506 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } }
I20260620 07:11:08.103644 30506 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 07:11:08.103701 30506 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 07:11:08.103790 30506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:08.104542 30506 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } }
I20260620 07:11:08.104828 30506 leader_election.cc:302] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb4088863f943fe8991752e901560e1; no voters:
I20260620 07:11:08.105073 30506 leader_election.cc:288] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 07:11:08.105170 30516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:08.105346 30516 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [term 1 LEADER]: Becoming Leader. State: Replica: bbb4088863f943fe8991752e901560e1, State: Running, Role: LEADER
I20260620 07:11:08.105659 30516 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } }
I20260620 07:11:08.106102 30506 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:08.107077 30518 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb4088863f943fe8991752e901560e1. Latest consensus state: current_term: 1 leader_uuid: "bbb4088863f943fe8991752e901560e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } } }
I20260620 07:11:08.107187 30518 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:08.107550 30525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:08.107789 30517 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb4088863f943fe8991752e901560e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb4088863f943fe8991752e901560e1" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 34631 } } }
I20260620 07:11:08.107905 30517 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:08.110028 30525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:08.115192 30525 catalog_manager.cc:1347] Generated new cluster ID: 9855d66f09ea46699d2b0fbac7bc0591
I20260620 07:11:08.115248 30525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:08.126720 30525 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:08.126987 30525 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:08.159588 30525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb4088863f943fe8991752e901560e1: Generated new TSK 0
I20260620 07:11:08.160413 30525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 07:11:08.165724 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30438
2026-06-20T07:11:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1278 ms)
[----------] 18 tests from AuthTokenIssuingTest (23231 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:08 dist-test-slave-wjw3 krb5kdc[30537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:08 dist-test-slave-wjw3 krb5kdc[30537](info): set up 2 sockets
Jun 20 07:11:08 dist-test-slave-wjw3 krb5kdc[30537](info): commencing operation
krb5kdc: starting...
W20260620 07:11:09.111375 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:09.218000 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35299
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:09.319542 30550 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:09.319780 30550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:09.319811 30550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:09.319870 30550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:09.323696 30550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:09.323752 30550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:09.323779 30550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:09.323798 30550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:09.323819 30550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:09.328732 30550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:35299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:09.329627 30550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:09.330529 30550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:09.338569 30556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:09.338565 30558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:09.338595 30555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:09.338634 30550 server_base.cc:1034] running on GCE node
I20260620 07:11:09.339370 30550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:09.340422 30550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:09.341576 30550 hybrid_clock.cc:648] HybridClock initialized: now 1781939469341563 us; error 25 us; skew 500 ppm
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:09.344151 30550 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:09.344986 30550 webserver.cc:460] Webserver started at http://127.8.24.62:34711/ using document root <none> and password file <none>
I20260620 07:11:09.345486 30550 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:09.345543 30550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:09.345752 30550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:09.347716 30550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "531d578b81e141ef8830ca9f8472e969"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "1caf9962e0fd9754a432355c84c4c4c2"
server_key_iv: "4d84537369c88ef15a87151f2ceda23f"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.348204 30550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "531d578b81e141ef8830ca9f8472e969"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "1caf9962e0fd9754a432355c84c4c4c2"
server_key_iv: "4d84537369c88ef15a87151f2ceda23f"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.351554 30550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260620 07:11:09.353660 30565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:09.354789 30550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 07:11:09.354900 30550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "531d578b81e141ef8830ca9f8472e969"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "1caf9962e0fd9754a432355c84c4c4c2"
server_key_iv: "4d84537369c88ef15a87151f2ceda23f"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.354984 30550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:09.375376 30550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:09.378242 30550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:09.378396 30550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:09.384774 30550 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:35299
I20260620 07:11:09.385052 30617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:35299 every 8 connection(s)
I20260620 07:11:09.385663 30550 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:09.388255 30618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:09.391727 30618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:09.392642 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30550
I20260620 07:11:09.392776 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:09.393039 8288 external_mini_cluster.cc:1428] Setting key 3685b348cad7bd7e8e181f76aeeeeee8
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:09.408325 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.393970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.8.24.62:35299)
0620 07:11:09.394483 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.394503 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.394633 (+ 130us) server_negotiation.cc:407] Connection header received
0620 07:11:09.395825 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.395830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.395997 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.396134 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.397081 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.397915 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.399270 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.399652 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.399759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.404070 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.404077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.404091 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.404117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.405570 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.406064 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.406068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.406069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.406159 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.406472 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.406475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.406476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.406817 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.406910 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.407316 (+ 406us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.407587 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":388,"spinlock_wait_cycles":51584,"thread_start_us":146,"threads_started":1}
I20260620 07:11:09.408327 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.393970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35299 (local address 127.0.0.1:38156)
0620 07:11:09.394471 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.394527 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.395540 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.396171 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.396177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.396609 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.396940 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.396959 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.398034 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.398037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.399167 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.399178 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.399793 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.399797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.399923 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.400450 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.400471 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.403918 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.405676 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.405686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.405695 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.405978 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.406234 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.406237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.406239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.406377 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.406921 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.406928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.407088 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.407276 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.407674 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":81,"thread_start_us":122,"threads_started":1}
I20260620 07:11:09.410854 30618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:09.411546 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.411301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60200)
0620 07:11:09.411403 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.411490 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 07:11:09.411712 30568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
I20260620 07:11:09.412461 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.412279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60210)
0620 07:11:09.412374 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.412414 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 07:11:09.412575 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40325: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260620 07:11:09.416311 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40325
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:11:09.461859 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } attempt: 1
I20260620 07:11:09.462702 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.462367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60212)
0620 07:11:09.462538 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.462610 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 07:11:09.463377 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.463153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60218)
0620 07:11:09.463249 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.463302 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 07:11:09.463521 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40325: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
W20260620 07:11:09.518256 30625 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:09.518493 30625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:09.518514 30625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:09.518550 30625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:09.522333 30625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:09.522377 30625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:09.522394 30625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:09.522414 30625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:09.522431 30625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:09.527261 30625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:40325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:09.528146 30625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:09.529000 30625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:11:09.531689 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } attempt: 2
I20260620 07:11:09.532464 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.532143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60226)
0620 07:11:09.532294 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.532374 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 07:11:09.533222 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.532983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60234)
0620 07:11:09.533102 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.533154 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 07:11:09.533352 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:40325: Network error: Client connection negotiation failed: client connection to 127.8.24.61:40325: connect: Connection refused (error 111)
W20260620 07:11:09.536888 30633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:09.536908 30631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:09.537046 30625 server_base.cc:1034] running on GCE node
W20260620 07:11:09.536871 30630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:09.537617 30625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:09.538564 30625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:09.539723 30625 hybrid_clock.cc:648] HybridClock initialized: now 1781939469539697 us; error 38 us; skew 500 ppm
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:09.542184 30625 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260620 07:11:09.543031 30625 webserver.cc:460] Webserver started at http://127.8.24.61:40633/ using document root <none> and password file <none>
I20260620 07:11:09.543576 30625 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:09.543643 30625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:09.543839 30625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:09.545802 30625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data/instance:
uuid: "fac6bb8bf1b2490cbe6c3014c540454e"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "e5a3d65c3511f20b6c31acde2352ae55"
server_key_iv: "e6ee8d4f27279a7bd466f4d0f3bd1f61"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.546314 30625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal/instance:
uuid: "fac6bb8bf1b2490cbe6c3014c540454e"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "e5a3d65c3511f20b6c31acde2352ae55"
server_key_iv: "e6ee8d4f27279a7bd466f4d0f3bd1f61"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.549692 30625 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 07:11:09.552044 30640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:09.553268 30625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:11:09.553386 30625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "fac6bb8bf1b2490cbe6c3014c540454e"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "e5a3d65c3511f20b6c31acde2352ae55"
server_key_iv: "e6ee8d4f27279a7bd466f4d0f3bd1f61"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.553475 30625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:09.565877 30625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:09.568282 30625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:09.568434 30625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:09.574590 30625 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:40325
I20260620 07:11:09.574810 30692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:40325 every 8 connection(s)
I20260620 07:11:09.575512 30625 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
I20260620 07:11:09.577991 30693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:09.580771 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30625
I20260620 07:11:09.580896 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-1/wal/instance
I20260620 07:11:09.581171 8288 external_mini_cluster.cc:1428] Setting key cf89fc761f3bd821461b86f40978847f
I20260620 07:11:09.581669 30693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:09.594842 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.583595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.8.24.62:35299)
0620 07:11:09.583736 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.583740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.584488 (+ 748us) server_negotiation.cc:407] Connection header received
0620 07:11:09.585399 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.585403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.585463 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.585560 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.586546 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.587358 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.588411 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.588703 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.588788 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.591740 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.591743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.591760 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.591784 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.593036 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.593487 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.593489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.593491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.593560 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.593853 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.593857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.593858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.594022 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.594103 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.594570 (+ 467us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.594688 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 07:11:09.595534 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.583915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35299 (local address 127.0.0.1:38166)
0620 07:11:09.584339 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.584387 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.585206 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.585678 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.585684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.585986 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.586408 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.586419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.587490 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.587493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.588304 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.588312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.588818 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.588821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.588945 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.589424 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.589438 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.591592 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.593143 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.593148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.593160 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.593390 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.593634 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.593637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.593639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.593763 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.594116 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.594122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.594341 (+ 219us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.594487 (+ 146us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.594766 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260620 07:11:09.596813 30693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260620 07:11:09.603799 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:43075
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:46605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:11:09.604430 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.597203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60248)
0620 07:11:09.597297 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.597309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.597377 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.597836 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.597838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.597950 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.598023 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.598027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.598920 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.598923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.599489 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.599499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.600020 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.600022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.600137 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.600523 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.600535 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.601595 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.603045 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.603049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.603051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.603267 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.603577 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.603579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.603580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.603649 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.604091 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.604093 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.604163 (+ 70us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.604237 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.604321 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":23}
I20260620 07:11:09.604646 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.597294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60248 (local address 127.8.24.61:40325)
0620 07:11:09.597534 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.597544 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.597561 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:11:09.597594 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.597597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.597740 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.597846 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.598102 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.598845 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.599573 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.599936 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.600021 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.601699 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.601707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.601716 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.601735 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.602949 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.603369 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.603375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.603376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.603490 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.603722 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.603725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.603726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.603992 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.604105 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.604326 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.604466 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260620 07:11:09.606417 30693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:09.607064 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.606825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58614)
0620 07:11:09.606943 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.607009 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 07:11:09.607223 30641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
I20260620 07:11:09.607971 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.607786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58628)
0620 07:11:09.607877 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.607922 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 07:11:09.608083 30693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
I20260620 07:11:09.624487 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } attempt: 3
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:11:09.633555 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.624938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60264)
0620 07:11:09.625066 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.625081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.625183 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.625411 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.625414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.625574 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.625673 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.625679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.626387 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.626391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.626923 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.626930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.627367 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.627371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.627488 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.628000 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.628016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.629950 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.631941 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.631945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.631947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.632200 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.632483 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.632488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.632490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.632597 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.633047 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.633050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.633148 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.633247 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.633388 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 07:11:09.633566 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.624974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.8.24.61:40325)
0620 07:11:09.625084 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.625088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.625139 (+ 51us) server_negotiation.cc:407] Connection header received
0620 07:11:09.625271 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.625274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.625314 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.625380 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.625779 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.626277 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.627022 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.627253 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.627359 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.630094 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.630098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.630100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.630125 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.631812 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.632296 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.632300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.632302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.632386 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.632702 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.632705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.632707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.632956 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.633043 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.633339 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.633441 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 07:11:09.635784 30618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:09.636513 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.636253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58632)
0620 07:11:09.636371 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.636427 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 07:11:09.637166 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.636926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58636)
0620 07:11:09.637062 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.637109 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260620 07:11:09.637295 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
I20260620 07:11:09.657292 30693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 1
I20260620 07:11:09.657917 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.657660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58642)
0620 07:11:09.657786 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.657843 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 07:11:09.658546 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.658337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58658)
0620 07:11:09.658440 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.658488 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 07:11:09.658651 30693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
I20260620 07:11:09.668440 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 1
I20260620 07:11:09.669000 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.668750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58668)
0620 07:11:09.668873 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.668928 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 07:11:09.669607 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.669402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58678)
0620 07:11:09.669500 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.669548 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 07:11:09.669719 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
W20260620 07:11:09.711154 30699 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:09.711391 30699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:09.711412 30699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:09.711480 30699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:09.715337 30699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:09.715380 30699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:09.715397 30699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:09.715415 30699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:09.715456 30699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:09.720360 30699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:46605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:35299,127.8.24.61:40325,127.8.24.60:43075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:09.721220 30699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:09.722093 30699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:11:09.726780 30693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 2
I20260620 07:11:09.727483 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.727194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58686)
0620 07:11:09.727323 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.727379 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 07:11:09.728111 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.727907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58688)
0620 07:11:09.728002 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.728048 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 07:11:09.728264 30693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
W20260620 07:11:09.729591 30706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:09.729579 30705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:09.729573 30708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:09.729792 30699 server_base.cc:1034] running on GCE node
I20260620 07:11:09.730347 30699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:09.731272 30699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:09.732419 30699 hybrid_clock.cc:648] HybridClock initialized: now 1781939469732411 us; error 50 us; skew 500 ppm
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:09.734850 30699 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260620 07:11:09.735638 30699 webserver.cc:460] Webserver started at http://127.8.24.60:39923/ using document root <none> and password file <none>
I20260620 07:11:09.736117 30699 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:09.736176 30699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:09.736358 30699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:09.738202 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data/instance:
uuid: "f34996ffbae846b9aeb38ee11603e13f"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "12226760d62e46d7fb79f1d145fe2c24"
server_key_iv: "69f00396cd3256f6eb4afc1db9f50827"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.738690 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal/instance:
uuid: "f34996ffbae846b9aeb38ee11603e13f"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "12226760d62e46d7fb79f1d145fe2c24"
server_key_iv: "69f00396cd3256f6eb4afc1db9f50827"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.741984 30699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260620 07:11:09.744405 30715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:09.744930 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 2
I20260620 07:11:09.745637 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.745355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58692)
0620 07:11:09.745490 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.745554 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 07:11:09.745795 30699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:11:09.745914 30699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "f34996ffbae846b9aeb38ee11603e13f"
format_stamp: "Formatted at 2026-06-20 07:11:09 on dist-test-slave-wjw3"
server_key: "12226760d62e46d7fb79f1d145fe2c24"
server_key_iv: "69f00396cd3256f6eb4afc1db9f50827"
server_key_version: "encryptionkey@0"
I20260620 07:11:09.746021 30699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:09.746310 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.746116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58698)
0620 07:11:09.746213 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.746257 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 07:11:09.746446 30618 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:43075: Network error: Client connection negotiation failed: client connection to 127.8.24.60:43075: connect: Connection refused (error 111)
I20260620 07:11:09.782485 30699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:09.785131 30699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:09.785276 30699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:09.791728 30699 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:43075
I20260620 07:11:09.791972 30767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:43075 every 8 connection(s)
I20260620 07:11:09.792637 30699 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
I20260620 07:11:09.795179 30768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:09.798599 30768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:09.799502 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30699
I20260620 07:11:09.799611 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781939200335300-8288-0/minicluster-data/master-2/wal/instance
I20260620 07:11:09.799867 8288 external_mini_cluster.cc:1428] Setting key 38084d4afc046cfdd153dbfb6fd4060e
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:11:09.811964 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.800449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.8.24.62:35299)
0620 07:11:09.800568 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.800572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.801463 (+ 891us) server_negotiation.cc:407] Connection header received
0620 07:11:09.802119 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.802122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.802167 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.802250 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.803584 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.804276 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.805506 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.805797 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.805894 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.808127 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.808131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.808133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.808158 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.810160 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810553 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.810557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.810559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.810640 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810910 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.810913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.810915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.811082 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.811165 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.811545 (+ 380us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.811812 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:11:09.812520 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.800879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:35299 (local address 127.0.0.1:38178)
0620 07:11:09.801319 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.801364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.801940 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.802986 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.802990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.803276 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.803451 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.803464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.804381 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.804385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.805407 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.805415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.805958 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.805962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.806067 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.806466 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.806486 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.808002 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.810243 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810259 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.810267 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.810464 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.810719 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.810724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.810826 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.811175 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.811181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.811310 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.811469 (+ 159us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.811812 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":82,"thread_start_us":113,"threads_started":1}
I20260620 07:11:09.812525 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.800880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.8.24.60:43075)
0620 07:11:09.801251 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.801264 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.801294 (+ 30us) server_negotiation.cc:407] Connection header received
0620 07:11:09.801926 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.801943 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.802138 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.802293 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.802938 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.804102 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.805210 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.805619 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.805754 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.808451 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.808456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.808465 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.808487 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.810039 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810420 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.810425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.810426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.810526 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.810760 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.810764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.810766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.811097 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.811262 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.811494 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.811750 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":263,"spinlock_wait_cycles":1920,"thread_start_us":131,"threads_started":1}
I20260620 07:11:09.813632 30768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:09.819393 30693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 3
I20260620 07:11:09.821120 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.814158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.8.24.61:40325)
0620 07:11:09.814300 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.814304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.814320 (+ 16us) server_negotiation.cc:407] Connection header received
0620 07:11:09.814362 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.814365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.814412 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.814482 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.814869 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.815419 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.816157 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.816375 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.816467 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.818316 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.818319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.818321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.818344 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.819556 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.819974 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.819978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.819979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.820053 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.820293 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.820297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.820298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.820494 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.820561 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.820866 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.820978 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:11:09.821529 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.814053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:40325 (local address 127.0.0.1:60272)
0620 07:11:09.814188 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.814199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.814267 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.814524 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.814527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.814680 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.814761 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.814765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.815537 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.815541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.816052 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.816059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.816466 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.816469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.816559 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.816974 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.816987 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.818195 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.819669 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.819672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.819675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.819881 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.820125 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.820128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.820130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.820202 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.820630 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.820633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.820705 (+ 72us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.821280 (+ 575us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.821396 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 07:11:09.822410 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.814565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.8.24.62:35299)
0620 07:11:09.814664 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.814668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.814680 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:11:09.814773 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.814775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.814811 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.814883 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.815359 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.815889 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.816591 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.816859 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.816951 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.818956 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.818959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.818962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.818986 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.820681 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.821084 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.821087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.821088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.821163 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.821435 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.821439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.821441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.821652 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.821867 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.822169 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.822272 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 07:11:09.822563 30768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:11:09.827016 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.819845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58718 (local address 127.8.24.60:43075)
0620 07:11:09.819942 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.819945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.820008 (+ 63us) server_negotiation.cc:407] Connection header received
0620 07:11:09.820128 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.820131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.820170 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.820240 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.820612 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.821170 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.822038 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.822444 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.822976 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.824476 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.824480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.824482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.824503 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.825655 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.826017 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.826020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.826022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.826090 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.826315 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.826325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.826327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.826472 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.826550 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.826802 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.826885 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 07:11:09.827023 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.819779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58718)
0620 07:11:09.819956 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.819969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.820046 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.820266 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.820269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.820415 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.820514 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.820520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.821280 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.821283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.821926 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.821935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.822569 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.822573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.822652 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.823098 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.823111 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.824373 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.825749 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.825752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.825754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.825937 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.826165 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.826168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.826170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.826243 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.826555 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.826558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.826663 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.826756 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.826879 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":1}
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:11:09.830377 30693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: Bootstrap starting.
I20260620 07:11:09.831840 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.823562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.8.24.61:40325)
0620 07:11:09.823646 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.823650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.823660 (+ 10us) server_negotiation.cc:407] Connection header received
0620 07:11:09.823690 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.823692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.823726 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.823790 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.824484 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.825207 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.826236 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.826544 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.826644 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.829113 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.829117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.829119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.829142 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.830377 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.830722 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.830725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.830727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.830802 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.831071 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.831074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.831076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.831227 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.831366 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.831606 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.831704 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":25}
I20260620 07:11:09.832531 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.822995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58720)
0620 07:11:09.823178 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.823190 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.823251 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.823945 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.823949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.824094 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.824174 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.824180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.825512 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.825515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.826261 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.826270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.826764 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.826768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.826832 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.827201 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.827213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.828990 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.830779 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.830782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.830784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.830995 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.831621 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.831624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.831625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.831701 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.832096 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.832099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.832184 (+ 85us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.832288 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.832396 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260620 07:11:09.832653 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.823161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.8.24.60:43075)
0620 07:11:09.823749 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.823757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.823768 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:11:09.823795 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.823798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.823829 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.823904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.824290 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.825419 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.826359 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.826673 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.826988 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.829116 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.829119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.829122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.829143 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.830654 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.831404 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.831406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.831407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.831532 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.831789 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.831792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.831793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.831998 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.832084 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.832354 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.832459 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":102,"threads_started":1}
I20260620 07:11:09.833096 30693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:09.833953 30693 log.cc:824] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:09.835054 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:09.835263 30768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: Bootstrap starting.
I20260620 07:11:09.836059 30693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: No bootstrap required, opened a new log
I20260620 07:11:09.836189 30592 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "531d578b81e141ef8830ca9f8472e969"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38188
I20260620 07:11:09.837563 30768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:09.838344 30768 log.cc:824] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:09.839797 30693 raft_consensus.cc:348] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.840000 30693 raft_consensus.cc:729] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac6bb8bf1b2490cbe6c3014c540454e, State: Initialized, Role: FOLLOWER
I20260620 07:11:09.840206 30768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: No bootstrap required, opened a new log
I20260620 07:11:09.840312 30693 consensus_queue.cc:260] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.840765 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.840891 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.841562 30693 sys_catalog.cc:564] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:09.843876 30768 raft_consensus.cc:348] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.844092 30768 raft_consensus.cc:729] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34996ffbae846b9aeb38ee11603e13f, State: Initialized, Role: FOLLOWER
I20260620 07:11:09.844393 30768 consensus_queue.cc:260] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.845530 30768 sys_catalog.cc:564] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: configured and running, proceeding with master startup.
W20260620 07:11:09.846357 30787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:09.846427 30787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:09.847488 30618 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } attempt: 3
I20260620 07:11:09.844873 30625 master_runner.cc:186] Error getting master registration for 127.8.24.62:35299: OK, instance_id { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" instance_seqno: 1781939469383035 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:11:09.848692 30625 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 07:11:09.851824 30800 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:09.851898 30800 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 20 07:11:09 dist-test-slave-wjw3 krb5kdc[30537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939469, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:11:09.849146 30699 master_runner.cc:186] Error getting master registration for 127.8.24.62:35299: OK, instance_id { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" instance_seqno: 1781939469383035 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:11:09.852900 30699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 07:11:09.853067 30789 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.853173 30789 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.856659 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.847865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.8.24.60:43075)
0620 07:11:09.847968 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.847972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.848028 (+ 56us) server_negotiation.cc:407] Connection header received
0620 07:11:09.848116 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.848119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.848157 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.848214 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.848700 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.849438 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.850528 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.850882 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.850964 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.853244 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.853248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.853250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.853274 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.854916 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.855339 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.855343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.855345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.855427 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.855719 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.855722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.855724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.855934 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.856024 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.856387 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.856507 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 07:11:09.856714 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.847809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:43075 (local address 127.0.0.1:58736)
0620 07:11:09.847959 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:09.847971 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:09.848050 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:09.848306 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:09.848308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:09.848485 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:09.848578 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.848585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.849587 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.849591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.850404 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:09.850414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.851091 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.851094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:09.851178 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.851652 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:09.851681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:09.853080 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:09.855015 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.855019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.855021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.855252 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.855533 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:09.855536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:09.855538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:09.855627 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:09.856038 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:09.856041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:09.856177 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 07:11:09.856318 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 07:11:09.856498 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260620 07:11:09.859793 30618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: Bootstrap starting.
I20260620 07:11:09.862226 30618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:09.863026 30618 log.cc:824] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:09.865553 30618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: No bootstrap required, opened a new log
I20260620 07:11:09.869699 30618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.870012 30618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531d578b81e141ef8830ca9f8472e969, State: Initialized, Role: FOLLOWER
I20260620 07:11:09.870443 30618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.870992 30803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.871171 30803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.872265 30592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 07:11:09.872375 30592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:09.873472 30592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
I20260620 07:11:09.872188 30618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:09.874349 30592 leader_election.cc:288] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [CANDIDATE]: Term 1 election: Requested vote from peers fac6bb8bf1b2490cbe6c3014c540454e (127.8.24.61:40325), f34996ffbae846b9aeb38ee11603e13f (127.8.24.60:43075)
I20260620 07:11:09.874918 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "531d578b81e141ef8830ca9f8472e969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f34996ffbae846b9aeb38ee11603e13f"
I20260620 07:11:09.874940 30667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "531d578b81e141ef8830ca9f8472e969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fac6bb8bf1b2490cbe6c3014c540454e"
I20260620 07:11:09.875131 30742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:09.875191 30667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:09.876297 30742 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 531d578b81e141ef8830ca9f8472e969 in term 1.
I20260620 07:11:09.876505 30667 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 531d578b81e141ef8830ca9f8472e969 in term 1.
I20260620 07:11:09.876680 30566 leader_election.cc:302] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 531d578b81e141ef8830ca9f8472e969, f34996ffbae846b9aeb38ee11603e13f; no voters:
I20260620 07:11:09.876979 30803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:09.877175 30803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [term 1 LEADER]: Becoming Leader. State: Replica: 531d578b81e141ef8830ca9f8472e969, State: Running, Role: LEADER
I20260620 07:11:09.877461 30803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } }
W20260620 07:11:09.879819 30571 tablet.cc:2369] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 07:11:09.880292 30814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:09.880405 30814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:09.882946 30803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 531d578b81e141ef8830ca9f8472e969. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.883044 30803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:09.893992 30698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.883751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.8.24.61:40325)
0620 07:11:09.883862 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.883866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.885314 (+ 1448us) server_negotiation.cc:407] Connection header received
0620 07:11:09.885487 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.885490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.885532 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.885578 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.886534 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.887316 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.888495 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.888832 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.888901 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.890622 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.890626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.890629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.890653 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.892233 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.892718 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.892722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.892724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.892832 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.893115 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.893118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.893120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.893310 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.893403 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.893720 (+ 317us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.893836 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 07:11:09.896134 30823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:09.897049 30823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:09.901175 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.884527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.8.24.62:35299)
0620 07:11:09.884625 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.884629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.891659 (+ 7030us) server_negotiation.cc:407] Connection header received
0620 07:11:09.891821 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.891824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.891866 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.891912 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.892975 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.893711 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.895916 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.896254 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.896352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.897902 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.897906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.897908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.897933 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.899486 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.899941 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.899944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.899946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.900028 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.900299 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.900302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.900304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.900496 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.900591 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.900921 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.901027 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:11:09.906625 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:09.885127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.8.24.60:43075)
0620 07:11:09.885222 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:09.885226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:09.892653 (+ 7427us) server_negotiation.cc:407] Connection header received
0620 07:11:09.894916 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:09.894919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:09.894975 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:09.895019 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:09.898997 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.899766 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.900851 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:09.901192 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:09.901260 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:09.903348 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:09.903352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:09.903355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:09.903380 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:09.904956 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.905392 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.905395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.905397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.905473 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:09.905742 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:09.905745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:09.905747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:09.905947 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:09.906050 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:09.906368 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 07:11:09.906476 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 07:11:09.916177 30742 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [term 1 FOLLOWER]: Refusing update from remote peer 531d578b81e141ef8830ca9f8472e969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:11:09.916177 30667 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [term 1 FOLLOWER]: Refusing update from remote peer 531d578b81e141ef8830ca9f8472e969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:11:09.916792 30803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:11:09.917237 30803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:11:09.920365 30789 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 531d578b81e141ef8830ca9f8472e969. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.920465 30789 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.920672 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 531d578b81e141ef8830ca9f8472e969. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.920774 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.925243 30803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.925349 30803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:09.926421 30789 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.926515 30789 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.926817 30826 mvcc.cc:205] Tried to move back new op lower bound from 7298824068770091008 to 7298824068621307904. Current Snapshot: MvccSnapshot[applied={T|T < 7298824068770091008}]
I20260620 07:11:09.927086 30828 mvcc.cc:205] Tried to move back new op lower bound from 7298824068770091008 to 7298824068621307904. Current Snapshot: MvccSnapshot[applied={T|T < 7298824068770091008}]
I20260620 07:11:09.927685 30815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.927803 30815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:09.928695 30776 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "531d578b81e141ef8830ca9f8472e969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "531d578b81e141ef8830ca9f8472e969" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 35299 } } peers { permanent_uuid: "fac6bb8bf1b2490cbe6c3014c540454e" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 40325 } } peers { permanent_uuid: "f34996ffbae846b9aeb38ee11603e13f" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 43075 } } }
I20260620 07:11:09.928797 30776 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:09.929389 30827 mvcc.cc:205] Tried to move back new op lower bound from 7298824068770091008 to 7298824068621307904. Current Snapshot: MvccSnapshot[applied={T|T < 7298824068770091008}]
I20260620 07:11:09.932382 30823 catalog_manager.cc:1347] Generated new cluster ID: b0dce4a6cb88483696149f0c9b4623f6
I20260620 07:11:09.932437 30823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:09.963495 30823 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:09.964720 30823 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:09.976563 30823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 531d578b81e141ef8830ca9f8472e969: Generated new TSK 0
I20260620 07:11:09.977365 30823 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:11:10.010394 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.007597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.8.24.61:40325)
0620 07:11:10.007716 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.007721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.009127 (+ 1406us) server_negotiation.cc:407] Connection header received
0620 07:11:10.009557 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.009562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.009611 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.009660 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.010268 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60306: BlockingRecv error: recv got EOF from 127.0.0.1:60306 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 07:11:10.011703 30622 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.008311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.8.24.62:35299)
0620 07:11:10.008414 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.008418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.010847 (+ 2429us) server_negotiation.cc:407] Connection header received
0620 07:11:10.011016 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.011019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.011067 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.011117 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.011588 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38200: BlockingRecv error: recv got EOF from 127.0.0.1:38200 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 07:11:10.012989 30773 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.008852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.8.24.60:43075)
0620 07:11:10.008976 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.008981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.012066 (+ 3085us) server_negotiation.cc:407] Connection header received
0620 07:11:10.012304 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.012307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.012355 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.012402 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.012871 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58758: BlockingRecv error: recv got EOF from 127.0.0.1:58758 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 07:11:10.017580 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.015895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.8.24.61:40325)
0620 07:11:10.016001 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.016005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.016410 (+ 405us) server_negotiation.cc:407] Connection header received
0620 07:11:10.016605 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.016609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.016674 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.016723 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.017493 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60312: BlockingRecv error: recv got EOF from 127.0.0.1:60312 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 07:11:10.022243 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.020873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.8.24.61:40325)
0620 07:11:10.020979 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.020983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.021459 (+ 476us) server_negotiation.cc:407] Connection header received
0620 07:11:10.021623 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.021626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.021670 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.021715 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.022160 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60314: BlockingRecv error: recv got EOF from 127.0.0.1:60314 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 07:11:10.027304 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.026004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.8.24.61:40325)
0620 07:11:10.026093 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.026097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.026547 (+ 450us) server_negotiation.cc:407] Connection header received
0620 07:11:10.026703 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.026706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.026747 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.026791 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.027220 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60330: BlockingRecv error: recv got EOF from 127.0.0.1:60330 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 07:11:10.034437 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.033096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.8.24.61:40325)
0620 07:11:10.033206 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.033210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.033667 (+ 457us) server_negotiation.cc:407] Connection header received
0620 07:11:10.033827 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.033830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.033871 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.033914 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.034355 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60338: BlockingRecv error: recv got EOF from 127.0.0.1:60338 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260620 07:11:10.045364 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.044329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.8.24.61:40325)
0620 07:11:10.044425 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.044429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.044662 (+ 233us) server_negotiation.cc:407] Connection header received
0620 07:11:10.044759 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.044762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.044811 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.044890 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.045268 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60340: BlockingRecv error: recv got EOF from 127.0.0.1:60340 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260620 07:11:10.064219 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.063138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.8.24.61:40325)
0620 07:11:10.063246 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.063250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.063410 (+ 160us) server_negotiation.cc:407] Connection header received
0620 07:11:10.063573 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.063576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.063620 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.063685 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.064113 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60350: BlockingRecv error: recv got EOF from 127.0.0.1:60350 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 07:11:10.098914 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.098001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.8.24.61:40325)
0620 07:11:10.098144 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.098148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.098241 (+ 93us) server_negotiation.cc:407] Connection header received
0620 07:11:10.098350 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.098352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.098397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.098463 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.098803 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260620 07:11:10.165756 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.164735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.8.24.61:40325)
0620 07:11:10.164862 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.164866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.164987 (+ 121us) server_negotiation.cc:407] Connection header received
0620 07:11:10.165126 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.165128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.165172 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.165238 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.165645 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60368: BlockingRecv error: recv got EOF from 127.0.0.1:60368 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 07:11:10.296715 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.295589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.8.24.61:40325)
0620 07:11:10.295727 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.295732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.295885 (+ 153us) server_negotiation.cc:407] Connection header received
0620 07:11:10.296036 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.296039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.296084 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.296152 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.296596 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60378: BlockingRecv error: recv got EOF from 127.0.0.1:60378 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260620 07:11:10.555619 30698 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:10.554551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.8.24.61:40325)
0620 07:11:10.554672 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:10.554676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:10.554753 (+ 77us) server_negotiation.cc:407] Connection header received
0620 07:11:10.554933 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:10.554935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:10.554981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:10.555042 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:10.555508 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60382: BlockingRecv error: recv got EOF from 127.0.0.1:60382 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 07:11:10.847702 30787 catalog_manager.cc:1259] Loaded cluster ID: b0dce4a6cb88483696149f0c9b4623f6
I20260620 07:11:10.847776 30787 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: loading cluster ID for follower catalog manager: success
I20260620 07:11:10.849447 30787 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: acquiring CA information for follower catalog manager: success
I20260620 07:11:10.850611 30787 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fac6bb8bf1b2490cbe6c3014c540454e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:11:10.852872 30800 catalog_manager.cc:1259] Loaded cluster ID: b0dce4a6cb88483696149f0c9b4623f6
I20260620 07:11:10.852922 30800 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: loading cluster ID for follower catalog manager: success
I20260620 07:11:10.854351 30800 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: acquiring CA information for follower catalog manager: success
I20260620 07:11:10.855298 30800 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f34996ffbae846b9aeb38ee11603e13f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:11:11.073150 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:11.069402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.8.24.61:40325)
0620 07:11:11.069675 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:11.069679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:11.069819 (+ 140us) server_negotiation.cc:407] Connection header received
0620 07:11:11.069951 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:11.069954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:11.070125 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:11.070230 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:11.070468 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.070976 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.071901 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.072114 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.072185 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:11.072404 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:11.072832 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:11.072915 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:11.072958 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 07:11:11.073002 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260620 07:11:11.078881 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:11.075897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.8.24.60:43075)
0620 07:11:11.076114 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:11.076117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:11.076129 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:11:11.076203 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:11.076205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:11.076307 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:11.076397 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:11.076565 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.077056 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.077764 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.077974 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.078034 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:11.078230 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:11.078617 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:11.078699 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:11.078725 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 07:11:11.078768 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260620 07:11:11.085255 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:11.081495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.8.24.61:40325)
0620 07:11:11.081804 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:11.081808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:11.081968 (+ 160us) server_negotiation.cc:407] Connection header received
0620 07:11:11.082042 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:11.082045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:11.082087 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:11.082158 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:11.082320 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.082987 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.084123 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.084462 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.084548 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:11.084850 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:11.084927 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:11.084990 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:11.085080 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 07:11:11.085128 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260620 07:11:11.085732 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:11.081616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.8.24.62:35299)
0620 07:11:11.081924 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:11.081927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:11.081938 (+ 11us) server_negotiation.cc:407] Connection header received
0620 07:11:11.082026 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:11.082029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:11.082175 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:11.082283 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:11.082490 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.083209 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.084320 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.084552 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.084638 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:11.084943 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:11.085258 (+ 315us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:11.085359 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:11.085425 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 07:11:11.085478 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":61,"threads_started":1}
I20260620 07:11:11.086076 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:11.081822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.8.24.60:43075)
0620 07:11:11.081932 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:11.081937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:11.082594 (+ 657us) server_negotiation.cc:407] Connection header received
0620 07:11:11.082694 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:11.082697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:11.082741 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:11.082802 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:11.083041 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.083656 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.084743 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:11.085001 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:11.085088 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:11.085663 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:11.085734 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:11.085798 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:11.085907 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 07:11:11.085954 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 07:11:11.088369 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30550
I20260620 07:11:11.094610 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30625
I20260620 07:11:11.099963 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30699
2026-06-20T07:11:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2935 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 07:11:11 dist-test-slave-wjw3 krb5kdc[30916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 07:11:11 dist-test-slave-wjw3 krb5kdc[30916](info): set up 2 sockets
Jun 20 07:11:11 dist-test-slave-wjw3 krb5kdc[30916](info): commencing operation
krb5kdc: starting...
W20260620 07:11:12.043531 8288 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/test-user.keytab.
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T07:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T07:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.24.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
Entry for principal kudu/127.8.24.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab.
I20260620 07:11:12.150780 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33195
--webserver_interface=127.8.24.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 07:11:12.251971 30929 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:12.252203 30929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:12.252224 30929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:12.252257 30929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:12.256067 30929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:12.256110 30929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:12.256127 30929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:12.256141 30929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:12.256155 30929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:12.261027 30929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.62.keytab
--principal=kudu/127.8.24.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.62:33195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.24.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:12.261883 30929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:12.262780 30929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:11:12.270377 30929 server_base.cc:1034] running on GCE node
W20260620 07:11:12.270315 30937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:12.270324 30934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:12.270334 30935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:12.271021 30929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:12.272022 30929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:12.273169 30929 hybrid_clock.cc:648] HybridClock initialized: now 1781939472273156 us; error 27 us; skew 500 ppm
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:12.279532 30929 init.cc:377] Logged in from keytab as kudu/127.8.24.62@KRBTEST.COM (short username kudu)
I20260620 07:11:12.280306 30929 webserver.cc:460] Webserver started at http://127.8.24.62:39251/ using document root <none> and password file <none>
I20260620 07:11:12.280792 30929 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:12.280853 30929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:12.281024 30929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:12.282922 30929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data/instance:
uuid: "a94986b083e24a1892a3a070f5092812"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "754786061dfdd292bc0857e171bfd272"
server_key_iv: "9e17df19b6939e6c0339aa29c0324aa5"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.283425 30929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance:
uuid: "a94986b083e24a1892a3a070f5092812"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "754786061dfdd292bc0857e171bfd272"
server_key_iv: "9e17df19b6939e6c0339aa29c0324aa5"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.286751 30929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 07:11:12.288960 30944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:12.290056 30929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:11:12.290158 30929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal
uuid: "a94986b083e24a1892a3a070f5092812"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "754786061dfdd292bc0857e171bfd272"
server_key_iv: "9e17df19b6939e6c0339aa29c0324aa5"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.290242 30929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:12.301697 30929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:12.304425 30929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:12.304589 30929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:12.311040 30929 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.62:33195
I20260620 07:11:12.311266 30996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.62:33195 every 8 connection(s)
I20260620 07:11:12.311926 30929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/data/info.pb
I20260620 07:11:12.314452 30997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:12.315230 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 30929
I20260620 07:11:12.315333 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-0/wal/instance
I20260620 07:11:12.315646 8288 external_mini_cluster.cc:1428] Setting key 5f6dac2c37d7f8b896227dcb5b95f858
I20260620 07:11:12.318645 30997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.61@KRBTEST.COM" created.
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:12.332646 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.320862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.8.24.62:33195)
0620 07:11:12.321236 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.321254 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.321481 (+ 227us) server_negotiation.cc:407] Connection header received
0620 07:11:12.322415 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.322420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.322594 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.322742 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.323538 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.324523 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.325637 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.325988 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.326098 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.328270 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.328277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.328290 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.328316 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.330032 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.330435 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.330441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.330452 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.330582 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.330825 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.330829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.330830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.331261 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.331392 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.331733 (+ 341us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.331943 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260620 07:11:12.332650 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.320863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33195 (local address 127.0.0.1:53788)
0620 07:11:12.321353 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.321388 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.322143 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.322764 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.322768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.323165 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.323387 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.323399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.324650 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.324653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.325528 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.325540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.326123 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.326127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.326248 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.326799 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.326814 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.328123 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.330125 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.330133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.330140 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.330341 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.330659 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.330661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.330662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.330750 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.331355 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.331359 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.331501 (+ 142us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.331629 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.331851 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"spinlock_wait_cycles":8064,"thread_start_us":124,"threads_started":1}
I20260620 07:11:12.334923 30997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:12.335577 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.335317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47110)
0620 07:11:12.335417 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.335514 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 07:11:12.335743 30946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
I20260620 07:11:12.336417 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.336251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47118)
0620 07:11:12.336336 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.336375 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 07:11:12.336534 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33209: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
Entry for principal kudu/127.8.24.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab.
I20260620 07:11:12.338883 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:33209
--webserver_interface=127.8.24.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 07:11:12.385730 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } attempt: 1
I20260620 07:11:12.386566 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.386280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47122)
0620 07:11:12.386423 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.386491 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 07:11:12.387179 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.387007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47132)
0620 07:11:12.387087 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.387127 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 07:11:12.387329 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33209: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
W20260620 07:11:12.441128 31004 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:12.441355 31004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:12.441375 31004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:12.441413 31004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:12.445222 31004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:12.445266 31004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:12.445282 31004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:12.445300 31004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:12.445313 31004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:12.450103 31004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.61.keytab
--principal=kudu/127.8.24.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.61:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.24.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:12.450917 31004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:12.451774 31004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 07:11:12.455476 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } attempt: 2
I20260620 07:11:12.456239 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.455970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47142)
0620 07:11:12.456112 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.456169 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 07:11:12.456833 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.456639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47148)
0620 07:11:12.456733 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.456777 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 07:11:12.456936 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.61:33209: Network error: Client connection negotiation failed: client connection to 127.8.24.61:33209: connect: Connection refused (error 111)
W20260620 07:11:12.459537 31009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:12.459597 31010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:12.459537 31012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:12.459617 31004 server_base.cc:1034] running on GCE node
I20260620 07:11:12.460209 31004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:12.461169 31004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:12.462321 31004 hybrid_clock.cc:648] HybridClock initialized: now 1781939472462303 us; error 31 us; skew 500 ppm
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:12.464824 31004 init.cc:377] Logged in from keytab as kudu/127.8.24.61@KRBTEST.COM (short username kudu)
I20260620 07:11:12.465564 31004 webserver.cc:460] Webserver started at http://127.8.24.61:46865/ using document root <none> and password file <none>
I20260620 07:11:12.466045 31004 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:12.466104 31004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:12.466269 31004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:12.468158 31004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data/instance:
uuid: "1f9a5462f4344cd19b01b3f7892feaf7"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "e2b3562594551bc1e771acd783015084"
server_key_iv: "72ef944510f1e5c62318e97073770b96"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.468650 31004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal/instance:
uuid: "1f9a5462f4344cd19b01b3f7892feaf7"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "e2b3562594551bc1e771acd783015084"
server_key_iv: "72ef944510f1e5c62318e97073770b96"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.471964 31004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 07:11:12.474118 31019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:12.475236 31004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 07:11:12.475337 31004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal
uuid: "1f9a5462f4344cd19b01b3f7892feaf7"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "e2b3562594551bc1e771acd783015084"
server_key_iv: "72ef944510f1e5c62318e97073770b96"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.475419 31004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:12.493649 31004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:12.496151 31004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:12.496301 31004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:12.502297 31004 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.61:33209
I20260620 07:11:12.502529 31071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.61:33209 every 8 connection(s)
I20260620 07:11:12.503149 31004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/data/info.pb
I20260620 07:11:12.503744 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 31004
I20260620 07:11:12.503863 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-1/wal/instance
I20260620 07:11:12.504159 8288 external_mini_cluster.cc:1428] Setting key c8997c0fbe7f31ebcd5b86fda92b7aae
I20260620 07:11:12.506253 31072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:12.509735 31072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.24.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.24.60@KRBTEST.COM" created.
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:12.522173 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.511411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.8.24.62:33195)
0620 07:11:12.511558 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.511561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.512328 (+ 767us) server_negotiation.cc:407] Connection header received
0620 07:11:12.513284 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.513288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.513348 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.513432 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.514474 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.515000 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.516129 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.516465 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.516567 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.518962 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.518965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.518967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.518991 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.520354 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.520897 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.520899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.520901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.520962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.521273 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.521275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.521276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.521440 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.521506 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.521952 (+ 446us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.522048 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 07:11:12.523026 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.511780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33195 (local address 127.0.0.1:53790)
0620 07:11:12.512196 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.512234 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.513071 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.513579 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.513585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.513889 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.514335 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.514347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.515113 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.515115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.516018 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.516028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.516604 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.516607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.516752 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.517345 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.517362 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.518798 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.520477 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.520483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.520497 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.520814 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.521033 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.521036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.521038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.521206 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.521510 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.521515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.521747 (+ 232us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.521896 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.522140 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260620 07:11:12.524340 31072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
Entry for principal kudu/127.8.24.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab.
I20260620 07:11:12.527801 8288 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
/tmp/dist-test-tasksJiSS1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:44419
--webserver_interface=127.8.24.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.24.20:36691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:11:12.532691 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.524739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47154)
0620 07:11:12.524838 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.524849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.524906 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.525539 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.525542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.525666 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.525734 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.525739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.526652 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.526655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.527212 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.527220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.527798 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.527801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.528051 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.528406 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.528418 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.529554 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.531354 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.531357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.531358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.531583 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.531845 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.531847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.531848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.531913 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.532350 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.532352 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.532428 (+ 76us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.532495 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.532583 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":159}
I20260620 07:11:12.532936 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.524842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.8.24.61:33209)
0620 07:11:12.525119 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.525126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.525143 (+ 17us) server_negotiation.cc:407] Connection header received
0620 07:11:12.525172 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.525175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.525310 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.525420 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.525832 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.526559 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.527305 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.527706 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.527791 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.529674 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.529682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.529691 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.529713 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.531258 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.531675 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.531682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.531684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.531769 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.531986 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.531989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.531991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.532265 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.532397 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.532598 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.532760 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
I20260620 07:11:12.534680 31072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:12.535288 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.535079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43724)
0620 07:11:12.535177 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.535239 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 07:11:12.535506 31020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.536264 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.536102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43736)
0620 07:11:12.536181 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.536220 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 07:11:12.536391 31072 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.548084 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } attempt: 3
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:11:12.556512 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.548593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.8.24.61:33209)
0620 07:11:12.548698 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.548701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.548714 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:11:12.548830 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.548832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.548870 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.548939 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.549362 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.549863 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.551369 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.551649 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.551729 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.553710 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.553713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.553715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.553736 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.554948 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.555342 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.555344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.555345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.555409 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.555770 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.555773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.555774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.555946 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.556026 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.556266 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.556364 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 07:11:12.556512 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.548529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47160)
0620 07:11:12.548650 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.548665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.548754 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.548980 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.548983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.549165 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.549276 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.549282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.549984 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.549987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.551266 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.551278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.551753 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.551757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.551861 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.552294 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.552308 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.553593 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.555042 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.555045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.555047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.555265 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.555522 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.555525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.555527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.555611 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.556034 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.556037 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.556125 (+ 88us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.556208 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.556334 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 07:11:12.557469 30997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } has no permanent_uuid. Determining permanent_uuid...
I20260620 07:11:12.558089 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.557886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43752)
0620 07:11:12.557991 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.558035 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 07:11:12.558733 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.558516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43760)
0620 07:11:12.558644 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.558684 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260620 07:11:12.558847 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.585595 31072 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 1
I20260620 07:11:12.586331 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.586074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43764)
0620 07:11:12.586203 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.586259 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 07:11:12.586942 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.586772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43780)
0620 07:11:12.586852 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.586892 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 07:11:12.587049 31072 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.589982 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 1
I20260620 07:11:12.590570 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.590357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43796)
0620 07:11:12.590467 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.590511 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 07:11:12.591161 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.590979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43808)
0620 07:11:12.591070 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.591112 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 07:11:12.591264 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
W20260620 07:11:12.635255 31078 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 07:11:12.635540 31078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 07:11:12.635573 31078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 07:11:12.635608 31078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 07:11:12.639456 31078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 07:11:12.639503 31078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 07:11:12.639521 31078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 07:11:12.639540 31078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 07:11:12.639559 31078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 07:11:12.644541 31078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.24.20:36691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.24.62:33195,127.8.24.61:33209,127.8.24.60:44419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/krb5kdc/kudu_127.8.24.60.keytab
--principal=kudu/127.8.24.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.24.60:44419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.24.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 06:29:27 UTC on ccycloud.quasar-mcsiyf.root.comops.site
build id 4457016
I20260620 07:11:12.645428 31078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 07:11:12.646345 31078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 07:11:12.654062 31085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 07:11:12.654057 31084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:12.654109 31078 server_base.cc:1034] running on GCE node
W20260620 07:11:12.654064 31087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 07:11:12.654700 31078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 07:11:12.655192 31072 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 2
I20260620 07:11:12.655834 31078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 07:11:12.655921 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.655631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43812)
0620 07:11:12.655775 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.655835 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 07:11:12.656872 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.656701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43824)
0620 07:11:12.656781 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.656821 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 07:11:12.656980 31072 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.657011 31078 hybrid_clock.cc:648] HybridClock initialized: now 1781939472656981 us; error 41 us; skew 500 ppm
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 07:11:12.659358 31078 init.cc:377] Logged in from keytab as kudu/127.8.24.60@KRBTEST.COM (short username kudu)
I20260620 07:11:12.660154 31078 webserver.cc:460] Webserver started at http://127.8.24.60:34371/ using document root <none> and password file <none>
I20260620 07:11:12.660646 31078 fs_manager.cc:359] Metadata directory not provided
I20260620 07:11:12.660701 31078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 07:11:12.660887 31078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 07:11:12.662727 31078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data/instance:
uuid: "82cbe3a9561841009853a20d70a364aa"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "069f93f99989d0c866f26b3b8f577776"
server_key_iv: "fc8d495b992903f031af6183b81a014b"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.663223 31078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal/instance:
uuid: "82cbe3a9561841009853a20d70a364aa"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "069f93f99989d0c866f26b3b8f577776"
server_key_iv: "fc8d495b992903f031af6183b81a014b"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.666389 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 2
I20260620 07:11:12.666635 31078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 07:11:12.667045 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.666761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43826)
0620 07:11:12.666896 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.666965 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 07:11:12.667721 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.667541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43840)
0620 07:11:12.667627 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.667670 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 07:11:12.667834 30997 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.24.60:44419: Network error: Client connection negotiation failed: client connection to 127.8.24.60:44419: connect: Connection refused (error 111)
I20260620 07:11:12.669034 31094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 07:11:12.670097 31078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 07:11:12.670199 31078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data,/tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal
uuid: "82cbe3a9561841009853a20d70a364aa"
format_stamp: "Formatted at 2026-06-20 07:11:12 on dist-test-slave-wjw3"
server_key: "069f93f99989d0c866f26b3b8f577776"
server_key_iv: "fc8d495b992903f031af6183b81a014b"
server_key_version: "encryptionkey@0"
I20260620 07:11:12.670287 31078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 07:11:12.690937 31078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 07:11:12.693353 31078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 07:11:12.693506 31078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 07:11:12.699510 31078 rpc_server.cc:307] RPC server started. Bound to: 127.8.24.60:44419
I20260620 07:11:12.699715 31146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.24.60:44419 every 8 connection(s)
I20260620 07:11:12.700340 31078 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/data/info.pb
I20260620 07:11:12.702780 31147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 07:11:12.702818 8288 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu as pid 31078
I20260620 07:11:12.702926 8288 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksJiSS1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781939200335300-8288-0/minicluster-data/master-2/wal/instance
I20260620 07:11:12.703173 8288 external_mini_cluster.cc:1428] Setting key 2cb5b9d3b3a3fae24cd84111a57d5d5c
I20260620 07:11:12.707157 31147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:12.715821 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.704285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.8.24.60:44419)
0620 07:11:12.704633 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.704647 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.704679 (+ 32us) server_negotiation.cc:407] Connection header received
0620 07:11:12.705259 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.705274 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.705594 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.705914 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.706648 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.707648 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.708439 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.708982 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.709116 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.711531 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.711537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.711547 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.711576 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.713162 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.713546 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.713550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.713553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.713653 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.713903 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.713905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.713907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.714314 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.714514 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.714686 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.714933 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":95,"threads_started":1}
I20260620 07:11:12.716995 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.708662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.8.24.62:33195)
0620 07:11:12.708788 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.708792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.709035 (+ 243us) server_negotiation.cc:407] Connection header received
0620 07:11:12.709150 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.709152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.709202 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.709279 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.709990 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.710451 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.711381 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.711702 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.711809 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.713864 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.713868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.713870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.713895 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.715366 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.715767 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.715770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.715772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.715855 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.716155 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.716159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.716160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.716411 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.716517 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.716802 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.716889 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 07:11:12.717000 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.708568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.62:33195 (local address 127.0.0.1:53794)
0620 07:11:12.708916 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.708959 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.709067 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.709329 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.709335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.709745 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.709874 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.709879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.710559 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.710562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.711273 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.711284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.711808 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.711812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.711952 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.712363 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.712378 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.713746 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.715483 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.715491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.715495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.715684 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.715950 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.715953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.715955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.716042 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.716505 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.716510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.716598 (+ 88us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.716699 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.716847 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":121,"threads_started":1}
I20260620 07:11:12.718156 31147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.62@KRBTEST.COM
I20260620 07:11:12.725574 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.718565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.61:33209 (local address 127.0.0.1:47170)
0620 07:11:12.718702 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.718713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.718774 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.719006 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.719008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.719153 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.719232 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.719237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.720191 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.720194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.720672 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.720679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.721103 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.721107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.721188 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.721491 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.721504 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.722603 (+ 1099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.724243 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.724246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.724248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.724460 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.724711 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.724714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.724715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.724792 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.725149 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.725151 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.725227 (+ 76us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.725320 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.725431 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260620 07:11:12.725625 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.718647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.8.24.61:33209)
0620 07:11:12.718751 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.718755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.718767 (+ 12us) server_negotiation.cc:407] Connection header received
0620 07:11:12.718858 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.718861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.718905 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.718984 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.719326 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.720092 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.720776 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.721000 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.721096 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.722731 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.722735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.722737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.722763 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.724145 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.724546 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.724549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.724551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.724630 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.724865 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.724868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.724870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.725068 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.725139 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.725375 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.725481 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 07:11:12.725932 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.717959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.8.24.62:33195)
0620 07:11:12.718057 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.718060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.718073 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:11:12.718137 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.718139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.718177 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.718250 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.718822 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.719493 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.720442 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.720649 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.720726 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.723148 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.723151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.723154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.723177 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.724332 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.724714 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.724717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.724719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.724785 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.725022 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.725024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.725025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.725189 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.725398 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.725511 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.725609 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:11:12.726240 31147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.60@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.24.61@KRBTEST.COM
I20260620 07:11:12.733515 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.726630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43868)
0620 07:11:12.726706 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.726718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.726781 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.727023 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.727026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.727168 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.727240 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.727246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.728048 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.728051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.728653 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.728662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.729059 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.729063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.729138 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.729520 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.729533 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.730612 (+ 1079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.732045 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.732048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.732050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.732209 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.732423 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.732426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.732428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.732496 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.733062 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.733064 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.733130 (+ 66us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.733226 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.733348 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 07:11:12.733610 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.726693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.8.24.60:44419)
0620 07:11:12.726786 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.726790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.726803 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:11:12.726837 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.726839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.726876 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.726942 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.727326 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.727952 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.728757 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.728974 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.729039 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.730729 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.730732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.730734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.730753 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.731958 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.732283 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.732286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.732288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.732349 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.732564 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.732567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.732568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.732704 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.732801 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.733279 (+ 478us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.733413 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":40}
I20260620 07:11:12.734283 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.726837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.8.24.61:33209)
0620 07:11:12.726928 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.726932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.726975 (+ 43us) server_negotiation.cc:407] Connection header received
0620 07:11:12.727077 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.727079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.727107 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.727157 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.727738 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.728290 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.729099 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.729341 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.729440 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.731488 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.731492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.731494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.731518 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.732696 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.733042 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.733044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.733045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.733103 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.733359 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.733361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.733363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.733548 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.733763 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.734079 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.734158 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 07:11:12.736874 31147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: Bootstrap starting.
I20260620 07:11:12.737166 8288 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 07:11:12.738132 30971 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a94986b083e24a1892a3a070f5092812"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53804
I20260620 07:11:12.739080 31147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:12.739781 31147 log.cc:824] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:12.741698 31147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: No bootstrap required, opened a new log
I20260620 07:11:12.745528 31147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.745755 31147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82cbe3a9561841009853a20d70a364aa, State: Initialized, Role: FOLLOWER
I20260620 07:11:12.746057 31147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.746515 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.746642 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.747156 31147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:12.748082 31072 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 3
W20260620 07:11:12.751950 31164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:12.752035 31164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.61@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
I20260620 07:11:12.751415 31078 master_runner.cc:186] Error getting master registration for 127.8.24.62:33195: OK, instance_id { permanent_uuid: "a94986b083e24a1892a3a070f5092812" instance_seqno: 1781939472309381 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:11:12.755059 31078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 07:11:12.756407 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.748530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.8.24.60:44419)
0620 07:11:12.748623 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.748628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.748643 (+ 15us) server_negotiation.cc:407] Connection header received
0620 07:11:12.748789 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.748792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.748854 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.748928 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.749369 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.749842 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.750693 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.750977 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.751068 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.753393 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.753397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.753400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.753425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.754925 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.755361 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.755365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.755366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.755478 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.755708 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.755711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.755713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.755871 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.755951 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.756178 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.756268 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 07:11:12.756407 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.748441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43870)
0620 07:11:12.748566 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.748582 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.748676 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.748952 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.748955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.749145 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.749255 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.749262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.749939 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.749942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.750591 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.750601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.751183 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.751187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.751268 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.751757 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.751773 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.753259 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.755041 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.755044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.755046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.755275 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.755551 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.755553 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.755556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.755634 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.755953 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.755956 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.756044 (+ 88us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.756124 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.756247 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 07:11:12.759465 31072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: Bootstrap starting.
I20260620 07:11:12.762123 31072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:12.762965 31072 log.cc:824] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:12.765015 31072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: No bootstrap required, opened a new log
I20260620 07:11:12.766952 30997 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } attempt: 3
I20260620 07:11:12.768837 31072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.769038 31072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f9a5462f4344cd19b01b3f7892feaf7, State: Initialized, Role: FOLLOWER
I20260620 07:11:12.769348 31072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.769879 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.770141 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.770601 31072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: configured and running, proceeding with master startup.
Jun 20 07:11:12 dist-test-slave-wjw3 krb5kdc[30916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781939472, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.24.62@KRBTEST.COM for kudu/127.8.24.60@KRBTEST.COM
W20260620 07:11:12.777550 31178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:12.777660 31178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:12.778182 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.767394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.8.24.60:44419)
0620 07:11:12.767513 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.767516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.767621 (+ 105us) server_negotiation.cc:407] Connection header received
0620 07:11:12.767723 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.767726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.767767 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.767828 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.768337 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.768809 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.769864 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.770175 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.770269 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.774358 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.774362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.774365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.774392 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.775969 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.776601 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.776604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.776606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.776700 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.777115 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.777119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.777120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.777335 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.777411 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.777908 (+ 497us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.778022 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 07:11:12.774009 31004 master_runner.cc:186] Error getting master registration for 127.8.24.62:33195: OK, instance_id { permanent_uuid: "a94986b083e24a1892a3a070f5092812" instance_seqno: 1781939472309381 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 07:11:12.778301 31004 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 07:11:12.778790 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.767345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.24.60:44419 (local address 127.0.0.1:43878)
0620 07:11:12.767552 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 07:11:12.767565 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 07:11:12.767654 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 07:11:12.767942 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 07:11:12.767945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 07:11:12.768132 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0620 07:11:12.768234 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.768241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.768919 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.768923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.769731 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 07:11:12.769742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.770389 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.770393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 07:11:12.770534 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.771683 (+ 1149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 07:11:12.771702 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 07:11:12.774206 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 07:11:12.776099 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.776102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.776104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.776453 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.776812 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 07:11:12.776815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 07:11:12.776817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 07:11:12.776973 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 07:11:12.777478 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 07:11:12.777481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 07:11:12.777642 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 07:11:12.778383 (+ 741us) client_negotiation.cc:240] Negotiation successful
0620 07:11:12.778585 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260620 07:11:12.781833 30997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: Bootstrap starting.
I20260620 07:11:12.784474 30997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: Neither blocks nor log segments found. Creating new log.
I20260620 07:11:12.785280 30997 log.cc:824] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: Log is configured to *not* fsync() on all Append() calls
I20260620 07:11:12.787391 30997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: No bootstrap required, opened a new log
I20260620 07:11:12.790268 30997 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.790472 30997 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94986b083e24a1892a3a070f5092812, State: Initialized, Role: FOLLOWER
I20260620 07:11:12.790786 30997 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.791260 31181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.791487 31181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.792261 30997 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 07:11:12.792337 30971 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 07:11:12.792431 30971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:12.793581 30971 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.794431 30971 leader_election.cc:288] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [CANDIDATE]: Term 1 election: Requested vote from peers 1f9a5462f4344cd19b01b3f7892feaf7 (127.8.24.61:33209), 82cbe3a9561841009853a20d70a364aa (127.8.24.60:44419)
I20260620 07:11:12.797180 31121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a94986b083e24a1892a3a070f5092812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "82cbe3a9561841009853a20d70a364aa"
I20260620 07:11:12.797412 31121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:12.798609 31121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94986b083e24a1892a3a070f5092812 in term 1.
I20260620 07:11:12.799278 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a94986b083e24a1892a3a070f5092812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f9a5462f4344cd19b01b3f7892feaf7"
I20260620 07:11:12.799510 31046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 07:11:12.800684 31046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94986b083e24a1892a3a070f5092812 in term 1.
I20260620 07:11:12.801708 30946 leader_election.cc:302] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f9a5462f4344cd19b01b3f7892feaf7, a94986b083e24a1892a3a070f5092812; no voters:
W20260620 07:11:12.805913 30950 tablet.cc:2369] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 07:11:12.807081 30929 master_runner.cc:263] No leader master found from master a94986b083e24a1892a3a070f5092812
I20260620 07:11:12.807137 30929 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 07:11:12.797386 31192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 07:11:12.807224 31192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 07:11:12.807336 31181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 07:11:12.807583 31181 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [term 1 LEADER]: Becoming Leader. State: Replica: a94986b083e24a1892a3a070f5092812, State: Running, Role: LEADER
I20260620 07:11:12.807914 31181 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } }
I20260620 07:11:12.809600 31181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94986b083e24a1892a3a070f5092812. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.809728 31181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:12.810068 31203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 07:11:12.810936 31203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 07:11:12.817235 31121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [term 1 FOLLOWER]: Refusing update from remote peer a94986b083e24a1892a3a070f5092812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:11:12.817901 31201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:11:12.821910 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94986b083e24a1892a3a070f5092812. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.822022 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.822956 31046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [term 1 FOLLOWER]: Refusing update from remote peer a94986b083e24a1892a3a070f5092812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 07:11:12.823519 31201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 07:11:12.825968 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.800354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.8.24.62:33195)
0620 07:11:12.800449 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.800453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.806328 (+ 5875us) server_negotiation.cc:407] Connection header received
0620 07:11:12.806533 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.806542 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.806589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.806637 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.808047 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.808808 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.810953 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.811287 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.811356 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.819158 (+ 7802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.819162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.819165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.819190 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.820961 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.824607 (+ 3646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.824611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.824613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.824696 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.825039 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.825043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.825045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.825265 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.825376 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.825689 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.825816 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 07:11:12.827087 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94986b083e24a1892a3a070f5092812. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.827189 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.832216 31201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.832327 31201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:12.833411 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.833521 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.833868 31204 mvcc.cc:205] Tried to move back new op lower bound from 7298824080652648448 to 7298824080624660480. Current Snapshot: MvccSnapshot[applied={T|T < 7298824080652648448}]
I20260620 07:11:12.836267 31181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.836364 31181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812 [sys.catalog]: This master's current role is: LEADER
I20260620 07:11:12.837327 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a94986b083e24a1892a3a070f5092812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94986b083e24a1892a3a070f5092812" member_type: VOTER last_known_addr { host: "127.8.24.62" port: 33195 } } peers { permanent_uuid: "1f9a5462f4344cd19b01b3f7892feaf7" member_type: VOTER last_known_addr { host: "127.8.24.61" port: 33209 } } peers { permanent_uuid: "82cbe3a9561841009853a20d70a364aa" member_type: VOTER last_known_addr { host: "127.8.24.60" port: 44419 } } }
I20260620 07:11:12.837432 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 07:11:12.838101 31207 mvcc.cc:205] Tried to move back new op lower bound from 7298824080652648448 to 7298824080624660480. Current Snapshot: MvccSnapshot[applied={T|T < 7298824080652648448}]
I20260620 07:11:12.840968 31205 mvcc.cc:205] Tried to move back new op lower bound from 7298824080652648448 to 7298824080624660480. Current Snapshot: MvccSnapshot[applied={T|T < 7298824080652648448}]
I20260620 07:11:12.841707 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.801409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.8.24.61:33209)
0620 07:11:12.801577 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.801581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.806549 (+ 4968us) server_negotiation.cc:407] Connection header received
0620 07:11:12.806975 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.806978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.807024 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.807072 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.812044 (+ 4972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.812817 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.817127 (+ 4310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.817463 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.817534 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.827209 (+ 9675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.827213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.827215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.827242 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.828820 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.831375 (+ 2555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.831378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.831380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.831482 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.840140 (+ 8658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.840144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.840146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.840370 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.840492 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.841428 (+ 936us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.841555 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260620 07:11:12.842177 31203 catalog_manager.cc:1347] Generated new cluster ID: 8c34c2803bda435fb37b64698c8659fb
I20260620 07:11:12.842231 31203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 07:11:12.842707 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:12.799773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.8.24.60:44419)
0620 07:11:12.799877 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.799893 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.806728 (+ 6835us) server_negotiation.cc:407] Connection header received
0620 07:11:12.809984 (+ 3256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.809987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.810036 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.810082 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.815215 (+ 5133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.815993 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.818161 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:12.818487 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:12.818558 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:12.829301 (+ 10743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 07:11:12.829305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 07:11:12.829308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 07:11:12.829336 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 07:11:12.830907 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.839748 (+ 8841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.839752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.839754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.839852 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 07:11:12.840787 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 07:11:12.840803 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 07:11:12.840805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 07:11:12.841009 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 07:11:12.841111 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:12.842455 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0620 07:11:12.842568 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 07:11:12.873797 31203 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 07:11:12.875154 31203 catalog_manager.cc:1504] Loading token signing keys...
I20260620 07:11:12.888204 31203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94986b083e24a1892a3a070f5092812: Generated new TSK 0
I20260620 07:11:12.889066 31203 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 07:11:12.922163 31150 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.920502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.8.24.60:44419)
0620 07:11:12.920618 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.920622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.921153 (+ 531us) server_negotiation.cc:407] Connection header received
0620 07:11:12.921329 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.921332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.921377 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.921425 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.922026 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43894: BlockingRecv error: recv got EOF from 127.0.0.1:43894 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 07:11:12.924314 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.922875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.8.24.61:33209)
0620 07:11:12.922985 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.922989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.923467 (+ 478us) server_negotiation.cc:407] Connection header received
0620 07:11:12.923651 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.923654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.923711 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.923761 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.924210 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47196: BlockingRecv error: recv got EOF from 127.0.0.1:47196 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260620 07:11:12.925644 31000 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.923080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.8.24.62:33195)
0620 07:11:12.923192 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.923196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.924884 (+ 1688us) server_negotiation.cc:407] Connection header received
0620 07:11:12.925026 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.925029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.925072 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.925116 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.925539 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53822: BlockingRecv error: recv got EOF from 127.0.0.1:53822 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 07:11:12.929266 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.927971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.8.24.61:33209)
0620 07:11:12.928061 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.928064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.928475 (+ 411us) server_negotiation.cc:407] Connection header received
0620 07:11:12.928651 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.928654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.928714 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.928757 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.929178 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47210: BlockingRecv error: recv got EOF from 127.0.0.1:47210 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260620 07:11:12.933866 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.932607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.8.24.61:33209)
0620 07:11:12.932687 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.932690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.933112 (+ 422us) server_negotiation.cc:407] Connection header received
0620 07:11:12.933253 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.933256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.933296 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.933337 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.933775 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47222: BlockingRecv error: recv got EOF from 127.0.0.1:47222 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 07:11:12.938390 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.937527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.8.24.61:33209)
0620 07:11:12.937614 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.937618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.937792 (+ 174us) server_negotiation.cc:407] Connection header received
0620 07:11:12.937878 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.937881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.937919 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.937987 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.938314 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47236: BlockingRecv error: recv got EOF from 127.0.0.1:47236 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 07:11:12.944802 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.943856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.8.24.61:33209)
0620 07:11:12.943989 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.943993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.944200 (+ 207us) server_negotiation.cc:407] Connection header received
0620 07:11:12.944283 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.944286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.944327 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.944388 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.944723 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47250: BlockingRecv error: recv got EOF from 127.0.0.1:47250 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260620 07:11:12.956036 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.955030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.8.24.61:33209)
0620 07:11:12.955134 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.955138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.955297 (+ 159us) server_negotiation.cc:407] Connection header received
0620 07:11:12.955421 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.955425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.955493 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.955553 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.955928 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47252: BlockingRecv error: recv got EOF from 127.0.0.1:47252 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260620 07:11:12.975419 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:12.974218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.8.24.61:33209)
0620 07:11:12.974394 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:12.974399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:12.974561 (+ 162us) server_negotiation.cc:407] Connection header received
0620 07:11:12.974757 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:12.974760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:12.974805 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:12.974870 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:12.975312 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47254: BlockingRecv error: recv got EOF from 127.0.0.1:47254 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260620 07:11:13.010353 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:13.009301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.8.24.61:33209)
0620 07:11:13.009443 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.009447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.009570 (+ 123us) server_negotiation.cc:407] Connection header received
0620 07:11:13.009709 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.009712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.009758 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.009824 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:13.010259 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47270: BlockingRecv error: recv got EOF from 127.0.0.1:47270 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260620 07:11:13.077302 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:13.076321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.8.24.61:33209)
0620 07:11:13.076468 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.076473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.076602 (+ 129us) server_negotiation.cc:407] Connection header received
0620 07:11:13.076717 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.076719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.076766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.076833 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:13.077188 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47282: BlockingRecv error: recv got EOF from 127.0.0.1:47282 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 07:11:13.208220 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:13.207270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.8.24.61:33209)
0620 07:11:13.207390 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.207393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.207539 (+ 146us) server_negotiation.cc:407] Connection header received
0620 07:11:13.207661 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.207663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.207709 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.207786 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:13.208122 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47296: BlockingRecv error: recv got EOF from 127.0.0.1:47296 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 07:11:13.467265 31077 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 07:11:13.466338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.8.24.61:33209)
0620 07:11:13.466464 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.466468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.466542 (+ 74us) server_negotiation.cc:407] Connection header received
0620 07:11:13.466661 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.466663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.466711 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.466786 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 07:11:13.467168 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47304: BlockingRecv error: recv got EOF from 127.0.0.1:47304 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 07:11:13.753242 31164 catalog_manager.cc:1259] Loaded cluster ID: 8c34c2803bda435fb37b64698c8659fb
I20260620 07:11:13.753321 31164 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: loading cluster ID for follower catalog manager: success
I20260620 07:11:13.754900 31164 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: acquiring CA information for follower catalog manager: success
I20260620 07:11:13.756004 31164 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82cbe3a9561841009853a20d70a364aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:11:13.778848 31178 catalog_manager.cc:1259] Loaded cluster ID: 8c34c2803bda435fb37b64698c8659fb
I20260620 07:11:13.778909 31178 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: loading cluster ID for follower catalog manager: success
I20260620 07:11:13.780557 31178 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: acquiring CA information for follower catalog manager: success
I20260620 07:11:13.781631 31178 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f9a5462f4344cd19b01b3f7892feaf7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 07:11:13.984599 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:13.981275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.8.24.61:33209)
0620 07:11:13.981496 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.981500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.981548 (+ 48us) server_negotiation.cc:407] Connection header received
0620 07:11:13.981667 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.981669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.981788 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.981893 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:13.982083 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.982600 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.983354 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.983605 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.983677 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:13.983904 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:13.984314 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:13.984412 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:13.984434 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 07:11:13.984479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":65,"threads_started":1}
I20260620 07:11:13.990317 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:13.987268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.8.24.60:44419)
0620 07:11:13.987500 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.987503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.987516 (+ 13us) server_negotiation.cc:407] Connection header received
0620 07:11:13.987631 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.987633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.987741 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.987830 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:13.987971 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.988464 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.989181 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.989389 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.989452 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:13.989654 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:13.990052 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:13.990135 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:13.990161 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 07:11:13.990204 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":70,"threads_started":1}
I20260620 07:11:13.996397 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:13.993200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.8.24.60:44419)
0620 07:11:13.993306 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.993310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.993458 (+ 148us) server_negotiation.cc:407] Connection header received
0620 07:11:13.993564 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.993566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.993610 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.993670 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:13.993869 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.994565 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.995488 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.995694 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.995773 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:13.996041 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:13.996099 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:13.996153 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:13.996237 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 07:11:13.996279 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 07:11:13.997539 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:13.992900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.8.24.61:33209)
0620 07:11:13.993103 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.993107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.994086 (+ 979us) server_negotiation.cc:407] Connection header received
0620 07:11:13.994185 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.994188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.994237 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.994301 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:13.994518 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.995151 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.996211 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.996527 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.996587 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:13.997115 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:13.997179 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:13.997221 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:13.997372 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 07:11:13.997419 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260620 07:11:13.997550 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 07:11:13.992897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.8.24.62:33195)
0620 07:11:13.993380 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0620 07:11:13.993385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 07:11:13.993399 (+ 14us) server_negotiation.cc:407] Connection header received
0620 07:11:13.993508 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 07:11:13.993511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 07:11:13.993690 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 07:11:13.993800 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 07:11:13.994002 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.994834 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.996061 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 07:11:13.996386 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 07:11:13.996474 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 07:11:13.996887 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 07:11:13.997201 (+ 314us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 07:11:13.997283 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 07:11:13.997351 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 07:11:13.997408 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":70,"threads_started":1}
I20260620 07:11:14.000373 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 30929
I20260620 07:11:14.006068 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 31004
I20260620 07:11:14.010905 8288 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksJiSS1/build/debug/bin/kudu with pid 31078
2026-06-20T07:11:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2911 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5846 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (273667 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST